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Chapter 1 



Overview and Summaries 
of Value-f or-money Audits 
and Reviews 



Overview 




BETTER INFORMATION HELPS ENSURE 
VALUE FOR MONEY 

There are numerous instances in this, my ninth 
Annual Report, where we noted that meaningful 
and reliable information was not being obtained or 
properly used to enhance the operation of govern- 
ment programs. In addition to enhancing day- to- 
day decision-making, better information would 
help government managers measure the results 
achieved for funds spent. It would also enable the 
Legislature and the public to reach knowledgeable 
conclusions about the extent to which their tax 
dollars have produced value for money. Areas we 
identified as needing better information include: 

Auto Insurance Regulatory Reform 

The Financial Services Commission of Ontario 
(FSCO) oversees auto insurance provided by the 
private sector. Auto insurance premiums are sig- 
nificantly higher in Ontario than anywhere else in 
Canada, and these high premium levels are driven 
primarily by high claims costs. However, FSCO does 
not get enough information to know whether insur- 
ance companies are handling claims judiciously and 
paying out the proper amounts. In addition, auto 
insurance fraud is estimated to account for up to 
15% of all claims in Ontario, and better informa- 



tion and more timely action are needed if both the 
government and the Commission are to proactively 
address this problem. 

Electricity Sector-Renewable Energy 
Initiatives 

The Green Energy and Green Economy Act, 2009 
gave the Minister of Energy authority to expedite 
the development of wind and solar power without 
many of the traditional planning and regulatory 
oversight processes. While this undoubtedly helped 
billions of dollars of renewable energy projects to 
move forward quickly, there are significant long- 
term costs associated with this initiative. Within 
this context, it will be critical for the Ministry of 
Energy and the Ontario Power Authority to object- 
ively measure the progress to date against the asso- 
ciated costs and provide policy options supported 
by sound underlying analyses that government 
decision-makers can use on a go-forward basis. 

Electricity Sector-Stranded Debt 

The restructuring of the electricity sector in 1999 
left Ontario Hydro with about $20 billion in what 
was called stranded debt — loans that its successor 
companies could not realistically service in the new, 
more competitive electricity market. Responsibility 
for servicing and managing this debt was given to 
the new Ontario Electricity Financial Corporation 
(OEFC) . It was expected that about $7.8 billion of 
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the debt could not be repaid by the electricity sector, 
and this debt was classified as "residual" stranded 
debt. The Electricity Act authorized collection of a 
Debt Retirement Charge from electricity consumers 
until this residual stranded debt was repaid. The 
Act requires the Minister of Finance to provide a 
public update "from time to time" on the outstand- 
ing balance of the residual stranded debt. The Debt 
Retirement Charge began in 2002 and the OEFC 
has collected more than $8 billion so far, but no 
update to the public has been provided. We believe 
this information should be provided to electricity 
consumers in the near future. 

Forest Management Program 

Private forestry companies that harvest timber are 
responsible for regenerating areas in which they 
operate, and the Ministry of Natural Resources is 
responsible to ensure that the companies carry out 
this work properly. The effectiveness of such over- 
sight is especially important because forests take 
upwards of 70 years to regenerate, which leaves 
little immediate financial incentive for private com- 
panies to undertake appropriate renewal activities. 
We found that while the Ministry has developed 
a good process to assess regeneration activities, it 
did not follow through in carrying out the required 
oversight work and thus did not have reliable infor- 
mation about the extent to which harvested areas 
were actually being successfully regenerated. 

Funding Alternatives for Family Physicians 

The province has traditionally paid family physicians 
on a fee-for-service basis. In recent years, however, 
the Ministry of Health and Long-Term Care has 
encouraged family doctors to participate in alternate 
funding arrangements designed to, among other 
things, improve patient access to care. While these 
arrangements have led to significant cost increases, 
the Ministry does not have adequate information 
to enable it to determine whether the new payment 
mechanisms have achieved their intended purposes. 



Funding Alternatives for Specialist 
Physicians 

The Ministry of Health and Long-Term Care offered 
alternate funding arrangements to specialist phys- 
icians to, for instance, encourage them to provide 
academic training and to work in more remote 
areas. While overall payments to specialists have 
increased significantly, as with family physicians, 
the Ministry has conducted little formal analysis 
to assess whether the expected benefits, including 
improved access to specialists by patients, have 
been achieved. 

Legal Aid Ontario 

Legal Aid Ontario has a mandate to provide legal 
assistance to Ontarians with little or no income. 
Although Ontario provides more funding for legal 
aid on a per capita basis than any other province, 
it issues the fewest certificates providing full legal 
representation on a per capita basis, which requires 
that more people rely on duty counsel and infor- 
mation from its call centre and website. Legal Aid 
Ontario, in conjunction with the Ministry of the 
Attorney General, has not conducted any formal 
analysis of the impact of this on low-income people 
needing legal representation. 

LCBO New Product Procurement 

The LCBO pays its suppliers a percentage of the 
retail selling price it wants to charge for their prod- 
ucts. Other provinces use a similar pricing mechan- 
ism. But if the LCBO obtained information on the 
lowest cost that the supplier was willing to accept, 
it could assess whether paying that cost would 
enable it to meet its retail price objectives while at 
the same time increasing its profit margins. 

Office of the Children's Lawyer 

Our research indicated that no other jurisdiction in 
Canada provides children with the same range of 
centralized legal services as Ontario's Office of the 
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Children's Lawyer, which is part of the Ministry of 
the Attorney General. The Office historically has 
exercised its discretion to refuse about 40% of child 
custody and access cases referred to it by the courts. 
However, it has never formally assessed the impact 
of these refusals on the children, their parents or 
guardians, and on the court system. 

Ontario Trillium Foundation 

The Ontario Trillium Foundation provides grants to 
hundreds of not-for-profit and charitable organiza- 
tions across Ontario, spending about $110 million 
each year. While it has a well-defined grant review 
and approval process, the supporting documenta- 
tion often did not demonstrate that the most worthy 
projects were funded for reasonable amounts; nor 
did the Foundation ensure that recipients always 
spent the grants for their intended purposes. 

Private Career Colleges 

The Ministry of Training, Colleges and Universities 
oversees 470 private career colleges serving 60,000 
students in Ontario. Five years ago, the Ministry 
stopped collecting information on graduation rates 
and employment success after graduation. Students 
who responded to our survey indicated that they 
would find such information extremely useful when 
evaluating colleges and their programs. Such data 
would also provide the Ministry with valuable 
information about the extent to which the colleges 
are meeting the needs of students and employers. 

Supportive Services for People with 
Disabilities 

The Ministry of Community and Social Services' 
Supportive Services program spent $571 million 
providing services to help people with develop- 
mental disabilities live at home and work in their 
communities. However, although the program 
relies on community agencies to deliver most of 
the services, the Ministry does not know whether 



the agencies are providing the appropriate level of 
service in return for the funding received; nor does 
it have reliable information on the level of unmet 
needs in each community across the province. 

Deja Vu 

On the first page of last year's Annual Report, I 
presented a few observations about the challenges 
facing Ontario over the next decade. I then wrote 
that I could not take credit for these pragmatic 
observations, as they were taken from reports 
issued by Ontario's Committee on Government 
Productivity — four decades ago. One of these 
observations from the early 1970s bears repeating 
because it aptly sums up my view that the better the 
information underlying a decision is, the better that 
decision is likely to be: "The core of sound decision- 
making is good information. In government, where 
decisions have far-reaching implications, the means 
of obtaining and effectively using information are 
of critical importance as tools for management." 

OUR WORK FOCUS 

Financial Audits 

The Legislature, the media, and the public usually 
pay the most attention to our value-for-money 
audits. However, doing financial audits remains one 
of our most critical legislative responsibilities. The 
purpose of these audits is to express opinions on 
whether the province's financial statements, as well 
as those of Crown agencies such as the LCBO, the 
Ontario Securities Commission, Legal Aid Ontario, 
and others, have been presented fairly. In the same 
way that corporate shareholders in the private sec- 
tor expect independent assurance that a company's 
financial statements fairly reflect its operating 
results and its balance sheet, Ontarians want the 
same assurances about public-sector entities. 

I am pleased to report that for the 18th straight 
year, the Office was able to provide assurance to 
the Legislature and the public that the government- 
prepared consolidated financial statements of 
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Ontario — the largest audited entity in the prov- 
ince — are fairly presented in accordance with Can- 
adian generally accepted accounting principles. The 
results of this work are discussed in Chapter 2. 

Similarly I can report that we concluded that 
the financial statements of all the Crown agencies 
we audited this year were also fairly presented. 

Value-for-money Audits 

About two-thirds of the Office's resources are 
devoted to the conduct of value-for-money audits. 
These audits focus on assessing the delivery of 
services to the public rather than being an audit of 
just the "numbers," as is the case with our financial 
audits. The next section in this chapter contains 
one-page summaries of the 14 value-for-money 
audits and reviews conducted this year. 

Pre-election Report on Ontario's Finances 

The Fiscal Transparency and Accountability Act, 
2004 (Act) requires the Minister of Finance to issue 
a report on Ontario's finances in advance of a prov- 
incial election to provide detailed information on 
the province's estimated future revenues, expenses, 
and projected surplus or deficit for the next three 
fiscal years. A key principle of the Act is that 
Ontario's fiscal policy be based on cautious assump- 
tions. Because a provincial general election had 
been called for October 6, 2011, the government 
released its 2011 Pre-Election Report on Ontario's 
Finances on April 26, 2011. 

The Act also requires the Auditor General to 
review this report to determine if it is reasonable, 
and to release a report on the results of this review. 
We released our review on June 28, 2011. 

Overall, we concluded that the government's 
estimates of revenues and interest on the public 
debt were based on prudent and cautious assump- 
tions. However, we also concluded that many of the 
assumptions underlying its estimates for program 
expenses (that is, expenses excluding interest on 
the public debt and reserves) were optimistic and 



aggressive rather than cautious, especially for public- 
sector salaries and health-care costs, which together 
account for the majority of total expenses. 

The Government Advertising Act 

The Government Advertising Act, 2004 requires our 
Office to review most proposed government adver- 
tising before the items are broadcast, published, or 
displayed. In the 2010/11 fiscal year, we reviewed 
1,082 individual advertising items. Chapter 5 con- 
tains a discussion of our work in this area. 

125th Anniversary of the Office 

On March 25, 1886, the Act to Provide for the Better 
Auditing of the Public Accounts of the Province was 
passed, creating an Office of the Provincial Auditor. 
That made 2011 our 125th anniversary. 

MPPs and past and present staff, including for- 
mer Auditors General Doug Archer and Erik Peters, 
attended an April reception at the Legislature. 

We also published a booklet on the Office's his- 
tory, along with brief sketches of the 12 Auditors 
General of the last 125 years and the Office's evolu- 
tion from accounting to accountability and from 
compliance auditing to value-for-money auditing. 

Some might think that 125 years of auditing 
would make for a pretty boring story. But I urge you 
to peruse this booklet. As you read it, I suspect you 
will be drawn by the narrative and the accompany- 
ing pictures that will take you back in time — and to 
some rather interesting times at that. 

The booklet is available at www.auditor.on.ca/ 
en/downloads_en/oago_anniversary_booklet.pdf. 





Summaries of Value-for- 
money Audits and Reviews 



The following are summaries of the value-for- 
money audits and the review reported in Chapter 3. 
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3.01 AUTO INSURANCE REGULATORY 
OVERSIGHT 

The Financial Services Commission of Ontario 
(FSCO), an arm's-length agency of the Ministry of 
Finance, is responsible for, among other things, regu- 
lating the province's insurance sector. FSCO's auto 
insurance activities include ruling on applications 
by private-sector insurance companies for changes 
in the premium rates that vehicle owners pay. FSCO 
must ensure that proposed premiums are justified 
based on such factors as an insurance company's 
past and anticipated claim costs and what would be 
a reasonable profit. FSCO also periodically reviews 
the statutory accident benefits available to people 
injured in auto accidents, and it provides dispute 
resolution services to settle disagreements between 
insurers and injured people about entitlement to 
statutory accident benefits. 

The government must balance the need for a 
financially stable auto insurance sector with ensur- 
ing that consumers pay affordable and reasonable 
premiums and receive fair and timely benefits and 
compensation after an accident. Claims payments 
are the largest driver of the cost of auto insurance 
premiums and, with the average cost of injury claims 
in Ontario being about $56,000 and five times more 
than the average injury claim in other provinces, 
Ontario drivers generally pay much higher premiums 
than other Canadian drivers do. However, claims 
costs in Ontario are also high because Ontario's 
coverage provides for one of the most comprehensive 
and highest benefit levels in Canada. 

The government has begun taking action to 
address the high cost of claims in Ontario. However, 
the following observations outline some of the chal- 
lenges FSCO faces if it is to be more successful in 
proactively fulfilling its role of protecting the public 
interest: 

• From 2005 to 2010, the total cost of injury 
claims under the Statutory Accident Benefits 
Schedule rose 150% even though the number 
of injury claims in the same period increased 
by only 30%. Benefit payments rose the most 



in the Greater Toronto Area, where drivers 
also generally pay much higher premiums. 

• FSCO had not routinely obtained assurances 
from insurance companies that they have paid 
the proper amounts for claims or that they 
have handled claims judiciously. Without such 
assurances, there is a risk that unnecessar- 
ily high payouts help insurers obtain FSCO 
approval for higher premium increases. 

• Industry estimates peg the value of auto insur- 
ance fraud in Ontario at between 10% and 15% 
of the value of 2010 premiums, or as much as 
$1.3 billion. Ontario does not have significant 
measures in place to combat fraud, and the 
government and FSCO are awaiting the rec- 
ommendations of a government-appointed 
anti-fraud task force expected in fall 2012. 

• In approving premium rates for individual 
insurance companies, FSCO allows insurers a 
reasonable rate of return — set at 12% in 1996, 
based on a 1988 benchmark long-term bond 
rate of 10%. However, that profit margin has 
not been adjusted downward, even though 
the long-term bond rate has been about 3% 
recently. Furthermore, FSCO needs to improve 
its documentation to demonstrate that it 
treats all insurers' premium-rate-change 
requests consistently and that its approvals 
are just and reasonable. 

• FSCO's mediation service is backlogged to 
the point that resolution of disputes between 
claimants and insurers is taking 10 to 12 
months, rather than the legislated 60 days. 

• The Motor Vehicle Accident Claims Fund, 
administered by FSCO to compensate people 
injured in auto accidents when there is no 
insurer to cover the claim, had $109 million 
less in assets as of March 31, 2011, than it 
needs to satisfy the estimated lifetime costs 
of all claims currently in the system. This 
unfunded liability is expected to triple by 
the 2021/22 fiscal year unless, for instance, 
the $15 fee currently added to every driver's 
licence renewal is doubled. 
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3.02 ELECTRICITY SECTOR- 
REGULATORY OVERSIGHT 

The Ontario Energy Board (Board) is charged with 
overseeing the electricity sector, which provides 
an essential commodity while operating as a near- 
monopoly The Board is responsible for protecting 
the interests of Ontario's 4.7 million electricity cus- 
tomers, and for helping to see that the sector is run 
efficiently and cost-effectively, and that it remains 
sustainable and financially viable. 

The Board has about 170 staff and operating 
costs of close to $35 million, all of which are paid by 
the entities that it regulates. The Board sets prices 
for electricity and its delivery, monitors electricity 
markets, and approves the administrative costs of 
the Ontario Power Authority and the Independent 
Electricity System Operator. 

Electricity prices for the average Ontario 
consumer have risen about 65% since the restruc- 
turing of the electricity sector in 1999, and prices 
are projected to rise another 46% in the next five 
years. In light of this, the Board's role of protecting 
consumers while setting rates that will provide a 
reasonable rate of return for the industry is all the 
more important. 

However, a number of factors limit the Board's 
ability to perform these duties to the extent that 
consumers and the electricity sector might expect. 
Our observations included the following: 

• The criterion that electricity bills be just and 
reasonable applies only to areas over which 
the Board has jurisdiction — only about half of 
the total charges on a typical bill. The Board 
can set rates only for the nuclear power and 
some of the hydro power produced by Ontario 
Power Generation (OPG), along with trans- 
mission, distribution, and certain other char- 
ges. The other half of a typical bill is based on 
government policy decisions over which the 
Board has no say, and these costs are not sub- 
ject to Board oversight. This includes the 50% 
of the electricity sold to residential customers 
that comes from other electricity suppliers 



and that, in total, constitutes 65% of the cost 
of the electricity component of the typical bill. 

• Consumers can purchase electricity through 
their utility at the Regulated Price Plan prices 
set by the Board or through an electricity 
retailer that sets its own price. About 15% of 
residential customers, looking for price stabil- 
ity on their power bills, signed fixed-price 
contracts with electricity retailers. These 
consumers could be paying 35% to 65% more 
for their electricity than they would pay had 
they not signed those contracts. In the last 
five years, the Board has received more than 
17,000 complaints from the public, the over- 
whelming majority of them about electricity 
retailers. Issues included misrepresentation 
by sales agents and forgery of signatures on 
contracts. Although the Board follows up on 
complaints, it has taken only a limited number 
of enforcement actions against retailers. 

• In areas in which it has jurisdiction, the Board 
sets rates using a quasi-judicial process that 
requires utilities and other regulated entities, 
such as OPG and Hydro One, to justify any 
proposed rate changes at a public hearing. 
Many small and mid-sized utilities say the cost 
of this process— $100,000 to $250,000 per 
application — can be as much as half the rev- 
enue increase sought in the first place. These 
costs, generally incurred every four years, are 
recovered from consumers. 

• Individuals or organizations wishing to 
participate in the hearings on behalf of con- 
sumers can obtain intervenor status, and can 
qualify for reimbursement of their expenses. 
However, many of the utilities and other 
regulated entities that have to reimburse the 
interveners say the number of requests that 
they receive can be onerous, the cost of pro- 
viding detailed information to the interveners 
is high, and they want the Board to better 
manage this process. 
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3.03 ELECTRICITY SECTOR- 
RENEWABLE ENERGY INITIATIVES 

The Ontario government has proposed that the 
province rely increasingly on renewable energy — 
especially wind and solar power. One reason for 
this is to help replace the power lost from the phas- 
ing out of coal-fired generation plants, to be com- 
pleted by 2014. In 2009, the government enacted 
the Green Energy and Green Economy Act (Act) to 
help attract investments and jobs in renewable 
energy, promote energy conservation, and reduce 
greenhouse gas emissions. 

The Ministry of Energy (Ministry) has developed 
programs and policies to implement the Act, and 
the Ontario Power Authority (OPA) has played 
a key role in planning and procuring renewable 
energy by contracting to buy power from develop- 
ers of renewable energy projects. Under the Act, the 
Minister is provided with the authority to supersede 
many of the government's usual planning and regu- 
latory oversight processes in order to expedite the 
development of renewable energy. 

Wind and solar power will add significant costs 
to ratepayers' electricity bills. It was felt that the 
higher costs associated with renewable energy were 
an acceptable trade-off given the environmental, 
health, and anticipated job-creation benefits. 
As well, these energy sources are not as reliable 
as traditional sources, and they require backup 
from alternative energy sources, such as gas-fired 
generation. 

Our significant observations relating to the 
implementation of the government's renewable 
energy policy included the following: 

• Ontario is on track to shut down its more 
than 7,500 megawatts (MW) — the capacity 
as of 2003 — of coal-fired generation by the 
end of 2014, to be replaced by nuclear power 
from refurbished plants, an increase of about 
5,000 MW of gas-fired generation, and renew- 
able energy, which is projected to increase to 
10,700 MW by 2018. 



« Because the Ministry and the OPA aimed to 
implement the Minister's directions as quickly 
as possible, no comprehensive evaluation was 
done on the impact of the billion-dollar com- 
mitment to renewable energy on such things 
as future electricity prices, net job creation or 
losses across the province, and greenhouse 
gas emissions. 

• When the Act was passed, the Ministry said 
implementing the Act would lead to modest 
increases in electricity bills of about 1% annu- 
ally. This was later increased to 7.9% annually 
over the next five years, with 56% of the 
increase due mainly to the cost of renewable 
energy. 

• The OPA was directed to replace a successful 
program — the Renewable Energy Standard 
Offer Program (RESOP) — with a much more 
costly Feed-in Tariff (FIT) program that 
required made-in-Ontario components and 
encouraged both larger and smaller genera- 
tion projects, but provided renewable energy 
generators with significantly more attractive 
contract prices than RESOP. 

• Although the OPA made a number of recom- 
mendations that could have significantly 
reduced the costs of FIT, these were held in 
abeyance until the two-year review of the 
FIT program could be undertaken so as to 
ensure price stability and maintain investor 
confidence. 

• A Korean consortium contracted by the Min- 
istry to develop renewable energy projects 
is to receive two additional incentives if it 
meets job-creation targets: $110 million in 
addition to the already attractive FIT prices; 
and priority access to Ontario's already 
limited transmission capacity. However, no 
economic analysis or business case was done 
to determine whether the agreement with the 
consortium was cost-effective, and neither 
the Ontario Energy Board nor the OPA was 
consulted about the agreement. 
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3.04 ELECTRICITY SECTOR-STRANDED 
DEBT 

The restructuring of Ontario's electricity sector in 
1999 and the creation of competitive wholesale 
and retail markets for electricity effective May 2002 
meant the province had to deal with the sector's 
stranded debt. This is defined as that portion of the 
total debt of the old Ontario Hydro that could not 
be serviced in a competitive market environment. 

On April 1, 1999, the Ministry of Finance deter- 
mined that Ontario Hydro's total debt and other lia- 
bilities stood at $38.1 billion. This greatly exceeded 
the estimated $17.2-billion market value of the 
assets being transferred to the five new companies 
that were formed to succeed Ontario Hydro. The 
resulting shortfall of $20.9 billion was determined 
to be "stranded debt." Responsibility for servicing 
and managing the stranded debt was given to the 
Ontario Electricity Financial Corporation (OEFC) . 

To service and retire the stranded debt, the 
government's long-term plan called for $13.1 bil- 
lion to be funded from expected dedicated revenue 
streams from the electricity sector, and for the 
remaining $7.8 billion — called the "residual 
stranded debt" — to be funded through a new Debt 
Retirement Charge (DRC), borne by electricity 
consumers. Since spring 2002, nearly every Ontario 
consumer's electricity bill has included the DRC. 

We have provided updates in past Annual 
Reports on the electricity sector's stranded debt, 
and this year we also reviewed the DRC, in response 
to ongoing interest shown by members of the Legis- 
lature, the public, and the media. 

Some of our observations included the 
following: 

• Progress in retiring the overall stranded debt 
has been slower than anticipated, due primar- 
ily to the lower-than-expected profitability of 
Hydro One and, particularly, Ontario Power 
Generation (OPG) . The lower their earnings, 
the lower the payments in lieu of taxes they 
are required to make to the OEFC. Some of the 
factors that have affected OPG's profitability 



over the past 11 years include cost overruns 
on electricity-generation projects, volatile 
investment returns, and public and political 
pressure to keep electricity rates low. 

• The original objective of the DRC, as stated 
by the then-Minister of Energy in 2000 and 
reiterated in the OEFC's 2010 and 2011 
Annual Reports, was that it would be paid by 
consumers until the residual stranded debt 
was retired. However, external legal advisers 
we engaged to assist us with our review of the 
DRC confirmed our view that section 85 of the 
Electricity Act, 1998 (Act), which is titled "The 
Residual Stranded Debt and the Debt Retire- 
ment Charge," allows the DRC to be used for 
any purpose that is in accordance with the 
OEFC's objectives and purposes, and not just 
the retirement of the residual stranded debt. 

• Section 85 requires that the Minister of 
Finance "from time to time" determine the 
amount of the outstanding residual stranded 
debt and make this determination public. 

To date, this has not been done. Given that 
the DRC has been collected from electricity 
consumers for almost a decade and that more 
than $8 billion in DRC revenue has been col- 
lected during that time, we believe that the 
Minister should make such a determination in 
the near future and make this determination 
public. We also suggest that the government 
consider specifying by regulation, as allowed 
for under section 85, how the amount of 
the outstanding residual stranded debt is to 
be calculated and when it is to be publicly 
reported. 
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3.05 FOREST MANAGEMENT PROGRAM 

Forests cover more than 700,000 square kilo- 
metres — about two-thirds — of Ontario. More than 
80% of these forests are on Crown land, and their 
management — harvesting, renewal, and mainten- 
ance — is governed mainly by the Crown Forest 
Sustainability Act, 1994 (CFSA). The CFSA aims to 
provide for the long-term sustainability of Ontario's 
Crown forests so that they meet the social, eco- 
nomic, and environmental needs of present and 
future generations. 

The forest industry is an important source of 
employment, especially in northern communities. 
In 2009, overall employment in the industry was 
estimated at 166,000 jobs, and the value of the sec- 
tor's products at $12 billion. However, the industry 
has experienced significant decline in recent years 
due mainly to the increase in value of the Canadian 
dollar and the economic downturn in the United 
States. Many mills have closed, resulting in a reduc- 
tion in timber harvest levels and associated forest 
management activities. 

Before the CFSA was enacted, the province was 
directly responsible for managing Crown forests, 
including regeneration. Under the CFSA, licensed 
forest management companies became responsible 
for overall forest sustainability planning and for 
carrying out all key management activities, includ- 
ing harvesting and forest renewal, on behalf of the 
Crown. The province's role in ensuring the sustain- 
ability of Crown forests has increasingly become 
one of overseeing the activities of the private-sector 
forest management companies. Such oversight is 
vital given that forests take upwards of 70 years to 
re-grow and these companies have little immediate 
financial incentive to carry out appropriate renewal 
activities. 

Overall, we found that improvements are 
needed if the Ministry of Natural Resources (MNR) 
and the Ministry of Northern Development, Mines 
and Forestry are to have adequate assurance about 
the long-term sustainability of Ontario's Crown 



forests. Our significant observations included the 
following: 

• In 2008/09 (the latest period for which 
information was available at the time of our 
audit), the two-thirds of the licensed forest 
management companies that had reported the 
results of their forest management activities 
indicated that although 93% of the total area 
that had been assessed by the companies had 
met the province's minimum 40% stocking 
standard, only 51% of the total area assessed 
had achieved silviculture success — a measure 
of whether the appropriate or preferred trees 
have grown back. 

• MNR's 40% stocking standard has not 
changed since the 1970s. Several other prov- 
inces hold the industry to higher standards. 

• Two factors ensure the likelihood of success- 
ful regeneration: preparing the site, not only 
before planting and seeding, but also before 
natural regeneration can take place; and then 
tending the site to kill off competing vegeta- 
tion. From 2004/05 to 2008/09, only about 

a third of the area targeted for regeneration 
was prepared and/or subsequently tended by 
the forest management companies. Several 
Independent Forest Audits completed in 2008 
and 2009 expressed concern about inad- 
equate site preparation or about non-existent 
or inadequate tending practices that were 
leading to reductions in growth, yield, and 
stand densities. 

• Although the Silviculture Effectiveness Mon- 
itoring program was a good initiative to assess 
forest industry regeneration efforts, we found 
that MNR's district offices were not complet- 
ing many of the "core tasks" in the program. 
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3.06 FUNDING ALTERNATIVES FOR 
FAMILY PHYSICIANS 

In the past, Ontario family physicians were trad- 
itionally paid almost entirely on a fee-for-service 
basis from the Ontario Health Insurance Plan for 
providing medical services. Over the past 10 years, 
the Ontario Ministry of Health and Long-Term 
Care (Ministry) has significantly increased its 
use of alternate funding arrangements for family 
physicians in order to, among other things, improve 
patients' access to care and provide income stability 
for physicians. 

By 2011, there were 17 types of alternate funding 
arrangements for family physicians. Under many 
of the funding arrangements, instead of receiving a 
fee for each service performed, physicians are paid 
an annual fee (called a capitation fee) to provide 
any of a specific list of services to each enrolled 
patient (that is, each patient who agrees to see the 
physician as his or her regular family physician) . 
Physicians may bill for additional services, as well 
as for services to non-enrolled patients, on a fee-for- 
service basis. In the 2010/11 fiscal year, the Family 
Health Group (FHG), Family Health Organization 
(FHO), and Family Health Network (FHN) arrange- 
ments accounted for more than 90% of family phys- 
icians in alternate funding arrangements, and more 
than 90% of enrolled patients. 

By the end of the 2009/10 fiscal year, more 
than 7,500 of the province's almost 12,000 family 
physicians were participating in alternate fund- 
ing arrangements, and more than nine million 
Ontarians had enrolled with these physicians. 
Of the $3.7 billion in total payments made to the 
province's family physicians in 2009/10, more than 
$2.8 billion was paid to physicians participating in 
alternate funding arrangements, with $1.6 billion 
of this amount related to non-fee-for-service pay- 
ments, such as annual capitation payments. 

In 2007/08, most family physicians participat- 
ing in alternate funding arrangements were being 
paid at least 25% more than their counterparts in 
the fee-for-service system. By 2009/10, the 66% 
of family physicians who participated in alternate 



funding arrangements were receiving 76% of the 
total amount paid to family physicians. The Min- 
istry has not tracked the full cost of each alternate 
funding arrangement since 2007/08, or analyzed 
whether the expected benefits of these more costly 
arrangements have materialized. 

Some of our other significant observations 
included the following: 

« Based on a survey it commissioned, the 
Ministry estimated that various initiatives, 
including alternate funding arrangements, 
have resulted in almost 500,000 more Ontar- 
ians having a family physician in 2010 than 
in 2007. However, the survey also found that 
patients generally indicated that the wait 
times to see a physician had not changed 
significantly. Although more than 40% of 
patients got in to see their physician within a 
day, the rest indicated that they had to wait up 
to a week or longer. 

• Of the 8.6 million patients enrolled with 
either an FHO or an FHG, 1.9 million (22%) 
did not visit their physician's practice in the 
2009/10 fiscal year, yet the physicians in these 
practices received $123 million just for having 
these patients enrolled. Further, almost half 
of these patients visited a different physician, 
and OHIP also paid for those visits. 

• The annual capitation fee for each patient 
enrolled in an FHO can be 40% higher than 
the annual fee for patients enrolled in an 
FHN, because almost twice as many servi- 
ces are covered under FHO arrangements. 
Nevertheless, in 2009/10, 27% of all services 
provided to FHO patients were not covered 
by the arrangement, and the Ministry paid 
an additional $72 million to the physicians 
for providing these services. Thirty percent 
of these services were for flu shots and Pap- 
smear technical services, yet the Ministry 
had not assessed whether it would be more 
cost-effective to have the annual capitation 
payment include coverage for these and other 
relatively routine medical services. 
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3.07 FUNDING ALTERNATIVES FOR 
SPECIALIST PHYSICIANS 

Specialist physicians provide services in more 
than 60 areas, including cardiology, orthopaedic, 
and emergency services, and obtain most of their 
income from fee-for-service OHIP billings. In the 
1990s, the Ministry of Health and Long-Term Care 
(Ministry) introduced alternate funding arrange- 
ments to encourage specialist physicians to provide 
certain services, such as training new physicians 
and doing research, as well as to encourage them 
to work in remote areas of the province. In 1999, 
the Ministry introduced specialist alternate funding 
arrangements for physicians who provide emer- 
gency services in hospitals. 

In the 2009/10 fiscal year, the Ministry paid 
almost $1.1 billion under specialist alternate fund- 
ing arrangements to more than 9,000 physicians, 
about 17% of the $6.3 billion the Ministry paid 
to all specialists that year and more than a 30% 
increase from 2006/07. As of March 31, 2010, half 
of the almost 13,000 specialists in the province and 
more than 90% of the 2,700 emergency department 
physicians received payments through a specialist 
alternate funding arrangement. 

We found that the Ministry has conducted little 
formal analysis of whether the alternate funding 
arrangements for specialists have yielded the 
expected benefits — such as improving patients' 
access to specialists — or whether the arrangements 
are cost-effective. We found, for instance, that 
payments to emergency department physicians 
increased by almost 40% between 2006/07 and 
2009/10, while the number of physicians working in 
emergency departments increased by only 10%, and 
the number of patient visits increased by only 7%. 

Some of our more significant observations were 
as follows: 

• There are numerous types of payments and 
premiums that specialists can earn under 
alternate funding arrangements, making it 
difficult for the Ministry to monitor contracts 
and related payments. For example, for aca- 



demic services at Academic Health Science 
Centres, there are as many as nine different 
categories of payments. 

• Ten Academic Health Science Centres 
received "specialty review funding" total- 
ling $19.7 million in 2009/10 to serve as an 
interim measure to alleviate shortages in five 
specialty areas. Yet similar interim funding 
has been given annually since 2002. 

• The Ministry paid $15,000 each to 234 north- 
ern specialists, who gave the Ministry permis- 
sion to collect information on each physician's 
income from provincial government-funded 
sources. 

• In order to monitor whether specialists funded 
under academic contracts performed the 
required services, the Ministry provided the 
specialists with a checklist to self-evaluate 
their performance. But the checklists were 
never requested back, and minimal other 
monitoring has been done. 

• In April 2008, the Ministry paid more than 
$15 million to 292 physicians who signed a 
document indicating that they intended to 
join a northern specialist alternate funding 
arrangement. However, 11 of the physicians, 
who were paid a total of $617,000, did not 
subsequently join such an arrangement, yet 
they were allowed to keep the funding. 
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3.08 LCBO NEW PRODUCT 
PROCUREMENT 

The mandate of the Liquor Control Board of 
Ontario (LCBO) — a Crown agency with the power 
to buy, import, distribute, and sell beverage alcohol 
products in Ontario — is to be a socially responsible, 
performance-driven, innovative, and profitable 
retailer. For the 2010/11 fiscal year, the LCBO's 
sales and other income totalled approximately 
$4.6 billion, and net income was $1.56 billion. The 
LCBO remitted virtually all that profit to the prov- 
ince. LCBO sales have increased 67% from 10 years 
ago, and its net income and the dividends it pays to 
the province have gone up 80% in that time. 

The LCBO offers consumers more than 21,000 
products available at more than 600 stores. It uses 
three methods to select and buy new products. The 
principal one, both for general-list products and for 
the Vintages fine wine and premium spirits line, is 
to issue a call to suppliers, known as a "needs let- 
ter," for a specific category of product. It can also 
buy products on an ad hoc basis, or, in the case of 
Vintages, directly from suppliers. 

The LCBO has the power to set the retail prices 
for the products it sells, guided by its mandate 
to promote social responsibility in the sale and 
consumption of alcohol while generating revenue 
for the province. Ontario's Liquor Control Act sets 
out minimum retail prices for alcohol to encourage 
social responsibility, and most Canadian jurisdic- 
tions operate this way. This means that the LCBO, 
like other Canadian jurisdictions, does not sell its 
products at the lowest possible prices, so retail 
prices for alcohol products are generally higher 
than those in the United States. 

Although some of the products that the LCBO 
sells are offered at lower prices in other Canadian 
jurisdictions, an April 2011 survey found that the 
LCBO had the lowest overall alcohol prices of all 
those jurisdictions, with the third-lowest prices for 
spirits and beer, and the lowest wine prices. 

Among our observations were the following: 



• In the private sector, large retailers use their 
buying power to negotiate lower costs with 
suppliers. However, the LCBO, despite being 
one of the largest purchasers of alcohol in the 
world, does not focus on getting the lowest 
cost it can for a product. Rather, the cost it 
pays is driven by the retail price it wants to 
charge for a product. The LCBO gives suppli- 
ers a price range within which it wants to sell 
a product. Suppliers' product submissions 
include, among other things, the retail price 
at which they want their product to sell in 
LCBO stores, and they then work backwards, 
applying the LCBO's fixed-pricing structure 
to determine their wholesale cost. We noted 
that in some instances suppliers submitted 
wholesale quotes that were significantly lower 
or higher than what the LCBO expected, in 
which cases the suppliers were asked to revise 
the amount of their quotes in order to match 
the agreed-upon retail price, which effectively 
either raises or lowers the price the LCBO pays 
the supplier for the product. 

• The LCBO does not negotiate volume dis- 
counts. This is also true of other Canadian 
jurisdictions we looked at. The LCBO's fixed- 
pricing structure gives it no incentive to nego- 
tiate lower wholesale costs; doing so would 
result in lower retail prices, and, in turn, 
lower profits, something that runs against 
the LCBO's mandate of generating profits for 
the province and encouraging responsible 
consumption. 

• We did note that the LCBO has many well- 
established purchasing practices that are 
consistent with those in Canadian and other 
jurisdictions. However, it could improve some 
of its processes relating to purchasing and 
monitoring of product performance to better 
demonstrate that these processes are carried 
out in a fair and transparent manner. 
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3.09 LEGAL AID ONTARIO 

Legal Aid Ontario is an independent corporation 
accountable to the Ministry of the Attorney General 
with a mandate to provide low-income people with 
consistently high-quality legal aid services in a cost- 
effective and efficient manner, while recognizing 
the private bar and clinics as the foundation for 
providing such services. 

Legal Aid Ontario provides assistance to people 
in three ways: it issues legal aid certificates to 
people who then retain private lawyers who in turn 
bill Legal Aid Ontario for those services; it pays and 
manages about 1,500 staff and contract lawyers to 
provide duty counsel services at criminal and family 
courts; and it funds and oversees 77 independent 
community legal clinics to assist people with gov- 
ernment assistance issues and tribunal representa- 
tion issues, such as landlord-tenant disputes. Legal 
Aid Ontario received $354 million in funding dur- 
ing the 2010/11 fiscal year, most of that from the 
provincial government. 

For at least the past decade, Ontario has spent 
more on legal aid support per capita than any other 
province, even though it has one of the lowest 
income eligibility thresholds and issues fewer cer- 
tificates entitling people to legal aid per capita than 
most other provinces. Legal Aid Ontario acknow- 
ledges the need to address a history of operating 
deficits, make its operations more cost-effective, 
improve access to its services, and help make the 
courts more efficient. We noted that it has a well- 
defined long-term strategy to address these issues 
and that it has moved to increase access to legal aid 
services beyond the issuing of certificates, such as 
through expanded use of duty counsel available at 
courthouses and through its new call centre. 

We felt that Legal Aid Ontario's multi-year 
long-term strategy was heading in the right direc- 
tion. However, the following are some of the areas 
the program must address if it is to be fully success- 
ful in meeting its mandate: 

• Only people with minimal or no income qual- 
ify for legal aid certificates or for assistance 



from community legal clinics, and the finan- 
cial eligibility cut-offs for qualifying have not 
changed since 1996 and 1993, respectively. 
This, combined with an escalation in the aver- 
age legal billing for each certificate issued, 
has meant fewer people over the last couple 
of years have been provided with certificates 
and more clients have been required to rely on 
duty counsel, legal advice, and information 
from Legal Aid Ontario's website for legal 
services. 

• Since its inception in 1999, Legal Aid Ontario 
has not had a quality assurance audit program 
in place with the Law Society of Upper Can- 
ada to help ensure that legal services provided 
by contract and staff lawyers to its low-income 
and vulnerable clients are of a high standard. 

• At the time of our audit, Legal Aid Ontario 
was working to address deficiencies with its 
lawyer payment system. Most importantly, 
strengthening of controls is required to ensure 
that all payments, which total $188 million 
annually, are justified. 

• Legal Aid Ontario's efforts to extract greater 
efficiencies from community legal clinics have 
strained its relationship with the clinics. 

• With the significant amount of solicitor-client 
privileged information on Legal Aid Ontario's 
information technology systems, we expected 
it to have performed recent and comprehen- 
sive privacy and threat risk assessments of its 
computer databases. However, the last privacy 
assessment was in 2004, and its systems have 
changed significantly since then. 

As with our 2001 audit, we again noted that 
Legal Aid Ontario was lacking key performance 
measures on the services it provides to its clients 
and stakeholders, and its annual reporting was 
three years overdue. 
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3.10 OFFICE OFTHE CHILDREN'S 
LAWYER 

The Office of the Children's Lawyer (Office), which 
is part of the Ministry of the Attorney General, 
provides children under the age of 18 with legal 
representation in child protection cases, custody 
and access cases, and property rights matters such 
as personal injury claims. Although the Office 
must provide legal representation for children in 
child protection cases and property rights cases 
when appointed by the court, it has discretion in 
accepting cases when the court requests its involve- 
ment in custody and access matters. 

The Office has approximately 85 staff, includ- 
ing lawyers, social workers, and support staff. The 
Office also engages what it calls "panel agents" — 
approximately 440 private lawyers and 180 clinical 
investigators — on an hourly fee-for-service basis. 
For the 2010/11 fiscal year, the Office's expendi- 
tures totalled approximately $32 million. It accepts 
about 8,000 new cases a year, and, as of March 31, 
2011, it had more than 11,000 open cases. 

Demand for the Office's legal and clinical 
investigation services is significant. The Office is 
unique — no other jurisdiction in Canada provides 
children with the same range of centralized legal 
services. Overall, the legal and investigative work 
done by the Office is valued by the courts, children, 
and other stakeholders. However, these services are 
often not assigned or delivered in a timely enough 
manner. 

We identified several areas in which the Office's 
systems, policies, and procedures needed improve- 
ment. Among our more significant findings: 
• The Office's case management system was 
not meeting its information needs, and the 
Office did not have an adequate process in 
place for evaluating the cost-effectiveness of 
its operations. For example, the Office had 
not adequately analyzed why its payments 
to panel agents had increased by more than 
$8 million, or 60%, over the last 10 years 
even though new cases accepted each year 



decreased by 20% and the Office's overall 
active caseload did not change significantly 
over the same period. 

• In the 2010/11 fiscal year, the Office exercised 
its discretion to refuse more than 40% of child 
custody and access cases referred to it by a 
court. We found, however, that the Office had 
not adequately assessed the impact of these 
refusals on the children and courts. Many of 
the decisions to refuse cases were made pri- 
marily because of a lack of financial resources. 

• Although the Office has substantially reduced 
the time it takes to accept or refuse custody 
and access cases — from 68 days in 2008/09 to 
39 days in 2010/11 — it still is not meeting its 
21 -day turnaround target. 

• In custody and access cases in which the 
Office is asked to investigate and then provide 
the court with a report and recommendations, 
Family Law Rules require it to do so within 90 
days. However, the Office met this deadline 
less than 20% of the time and did not have 
any formal strategy in place for improving its 
performance in this regard. 

• The Office had a sound process for ensuring 
that personal rights lawyers and clinical inves- 
tigators were well qualified and selected fairly. 
However, there was no open selection process 
in place for the almost 100 property rights 
lawyers the Office engaged. 

• The Office permits property rights panel 
lawyers to charge up to $350 an hour when 
recovering their costs from a child's estate, or 
from trust or settlement funds. Yet if the same 
lawyers charge their services directly to the 
Office, they are paid $97 an hour. 

• The Office's programs for reviewing the qual- 
ity of the work performed by panel agents did 
not include an assessment of whether the fees 
charged were reasonable. 
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3.11 ONTARIO TRILLIUM FOUNDATION 

The Ontario Trillium Foundation (Foundation) was 
established in 1982 as an agency of the Ontario gov- 
ernment. Its mission is to build "healthy and vibrant 
communities throughout Ontario by strengthening 
the capacity of the voluntary sector, through invest- 
ments in community-based initiatives." 

In the 2010/11 fiscal year, the Foundation 
distributed about 1,500 grants worth more than 
$110 million to not-for-profit and charitable organ- 
izations working in the areas of human and social 
services, arts and culture, environment, and sports 
and recreation. Most of the grant money goes to 
pay the salaries and wages of people working in 
these organizations. 

The agency has a volunteer board of directors 
and about 120 full-time staff located at its Toronto 
head office and in 16 regional offices. In addi- 
tion, more than 300 volunteers may be named to 
grant-review teams across the province — 18 to 24 
volunteers on each team — to vote on which projects 
or organizations should be funded. 

Among our observations are the following: 

• One of the Foundation's main responsibilities 
is to ensure that it gives out its allocation of 
more than $100 million each year to commun- 
ity not-for-profit and charitable organizations. 
A wide range of projects can be funded, as 
long as they support the local community 
and relate to the areas mentioned above. The 
determination of value for money received 
for each grant may well be in the eye of the 
beholder, and it is within this context that the 
Foundation operates. 

• Although the Foundation has a well-defined 
grant application and review process for 
deciding which applicants receive grants, the 
underlying process and resulting documenta- 
tion often did not demonstrate that the most 
worthy projects were funded for reasonable 
amounts. This was due to the fact that there 
was often little documentation available to 
demonstrate that the Foundation objectively 



compared the relative merits of different 
proposals, adequately assessed the reason- 
ableness of the grant amounts requested and 
approved, and effectively monitored and 
assessed spending by recipients. 

• Many of the grant recipients we visited could 
not substantiate the expenditure and per- 
formance information they reported to the 
Foundation. 

• We felt the Foundation's website was compre- 
hensive and informative. However, the Foun- 
dation could do more to inform community 
organizations about the availability of grants 
and about the application process. It could, 
for example, consider advertising in local and 
ethnic-community newspapers. 

• Although the Foundation's administra- 
tive expenditures are relatively modest, it 
nevertheless needs to tighten up certain of its 
administrative procedures to ensure that it 
complies with the government's procurement 
and employee-expense guidelines. 
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3.12 PRIVATE CAREER COLLEGES 

Private career colleges are independent organiza- 
tions that offer certificate and diploma programs 
in fields such as business, health services, and 
information technology. They often cater to adult 
students who need specific job skills to join the 
workforce or become more competitive in the job 
market. There are about 470 registered private 
career colleges in Ontario serving 60,000 students. 

The Ministry of Training, Colleges and Univer- 
sities (Ministry) administers the Private Career 
Colleges Act, 2005 (Act). The Act focuses on pro- 
tecting the rights of students. Through the Training 
Completion Assurance Fund, the Act also provides 
students with the right to complete their training at 
another institution at no additional cost if the col- 
lege they are attending ceases operations. 

Although the Ministry does not fund private 
career colleges directly, it provides significant fund- 
ing to the sector through its employment training 
and student assistance programs. Over the past 
three fiscal years, a total of almost $350 million 
was provided through the Ministry's Second Career 
and Skills Development programs to an average 
of 13,000 students annually for their tuition at 
private career colleges. In addition, in the last three 
academic years, almost $200 million in loans and 
grants was provided to an annual average of 9,500 
students through the Ministry's Ontario Student 
Assistance Program. 

The Ministry has recently undertaken a number 
of good initiatives to improve its oversight of private 
career colleges and strengthen protection for stu- 
dents. However, further improvements are needed 
to ensure compliance with the Act, its regulations, 
and ministry policies, and to protect students. 
The following are some of our more significant 
observations: 

• Although it has taken steps to identify and act 
on unregistered colleges, the Ministry could 
make better use of information it already has 
on hand to identify colleges that continue to 
operate illegally. For example, the Ministry 



does not routinely check to see that schools 
that have been closed remain closed. We 
reviewed a sample of schools that had been 
closed and found that a number appeared to 
be offering courses. 

• In 2006, the Ministry stopped collecting infor- 
mation on graduation rates and employment 
upon graduation for private career colleges, 
something it does for public colleges. More 
than 85% of the private career college gradu- 
ates who responded to our survey said that 
such student outcome data would have been 
useful in helping them with their choice of 
college and courses. 

• The Ministry did not have adequate processes 
in place for assessing the financial viability of 
colleges when they seek to renew their annual 
registration. One college with significant 
losses had its registration renewed without 
any evidence that its financial viability had 
been reviewed. The college subsequently 
closed, costing the Training Completion 
Assurance Fund more than $800,000. 

• The Ministry can enter and inspect the prem- 
ises of a registered private career college or 
an unregistered institution that should be 
registered. Although a recent risk assessment 
done by the Ministry identified 180 private 
career college campuses with multiple com- 
pliance risk factors, the Ministry could not 
demonstrate that it had done enough inspec- 
tions to manage the risk of non-compliance 
with the Act and its regulations. There are 
approximately 470 registered private career 
colleges with 650 campuses in Ontario, but 
the Ministry estimated that only 30 campuses 
had been inspected in 2010. 
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3.13 STUDENT SUCCESS INITIATIVES 

Ontario's Student Success Strategy is a collection 
of initiatives that has been implemented by the 
Ministry of Education (Ministry) since 2003 to help 
secondary school students graduate with their high 
school diplomas. A 2003 Ministry report cited the 
graduation rate at the time as 68%. The overall 
objective of the Student Success Strategy was to 
reduce the high school dropout rate, and raise the 
graduation rate to 85% by the 2010/11 school year. 

The Ministry's Student Achievement Division 
is responsible for developing and monitoring the 
Student Success Strategy while school boards 
and schools are responsible for delivering the 
strategy's initiatives. Every board receives funding 
for one student success leader to help implement 
programs in its schools, as well as funding for one 
student success teacher per secondary school who 
is responsible for providing supports to students 
at risk of not graduating. In the 2010/11 school 
year, the Ministry provided almost $130 million to 
school boards for the delivery of student success 
initiatives. 

Steady progress has been made toward the goal 
of an 85% graduation rate, and the rate stood at 
81% for the 2009/10 school year. However, we did 
note some areas where refinements to the initia- 
tives would help ensure that the Ministry's object- 
ives can be met and that students have acquired 
the knowledge and skills they need to go on to 
post-secondary education or employment. Some of 
our observations were as follows: 

• Ontario school boards we visited track risk 
factors such as gender, absenteeism, and 
course success to help identify students at risk 
and then provide them with supports. How- 
ever, we noted that some other jurisdictions 
have found that targeting supports to specific 
groups of students based on factors such as 
ethnicity, disability, and economic status has 
been very effective in improving graduation 
rates. For example, targeted programming in 
one U.S. high school resulted in a 92% gradu- 



ation rate for African-American students, 
which far exceeded the state-wide average of 
67% for this group. 

• The Ministry's reported graduation rate is 
based on calculating the percentage of grade 9 
students who graduate within five years. How- 
ever, the 2009/10 graduation rate would have 
been 72% if it had been based on graduation 
within the four-year span of high school. On 
the other hand, the graduation rate would 
have been 91% if it had been extended to 
when students reach the age of 25. 

• The Ministry relies primarily on tracking 
changes in the graduation rate to measure 
the outcome of the Student Success Strategy. 
However, graduation rates are generally not 
publicly available by school board, and boards 
do not use a consistent method of calculating 
graduation rates, so it is difficult to meaning- 
fully compare rates across the province. Better 
information is also needed on graduates' level 
of preparedness for post-secondary studies 
and employment. 

• We noted situations where the work place- 
ments in the Cooperative Education program 
did not appear to complement the students' 
curriculum requirements for in-class learning. 
Students earned credits in a wide range of 
placements, such as clothing stores, fast-food 
outlets, coffee shops, and laboratories. 

• In the 2009/10 and 2010/11 school years, 
only $15 million of the $245 million the 
Ministry provided to school boards for student 
success initiatives was allocated based on a 
direct assessment of student needs. Much of 
the remaining funding was allocated based on 
the number of students in each board, rather 
than being targeted to the boards, schools, 
and students most in need of support. 
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3.14 SUPPORTIVE SERVICES FOR 
PEOPLE WITH DISABILITIES 

The Ministry of Community and Social Services 
(Ministry) funds a variety of supportive services 
programs to help people with developmental dis- 
abilities live at home and work in their commun- 
ities. In 2010/11, the Ministry spent $571 million 
on such programs, $472 million of that through 
412 contracts with transfer-payment agencies in 
nine regions, which provided services to about 
134,000 eligible people. The Ministry-administered 
Special Services at Home (SSAH) program received 
$99 million to serve 24,000 families. 

Agencies that receive transfer-payment funding 
provide or arrange for such services as assessment 
and counselling, speech and language therapy, 
behaviour intervention therapy, and respite care. 
Agencies also administer the Passport program, 
which provides direct funding to families for such 
things as personal development, as well as social 
and recreational activities. The SSAH program 
provides direct funding to eligible families for 
purchasing supports and services beyond those 
typically provided by families and that are designed 
primarily to enhance personal development and 
provide family relief through respite care. 

Many of the concerns noted in our last audit 
of this program 15 years ago have still not been 
satisfactorily addressed. The Ministry still does not 
have adequate assurance that its service delivery 
agencies are providing an appropriate and consist- 
ent level of support in a cost-effective manner to 
people with developmental disabilities. The Min- 
istry's oversight procedures are still not adequate 
to ensure that quality services are provided and 
that public funds are properly managed by transfer- 
payment agencies. 

Although the Ministry is in the midst of a com- 
prehensive Developmental Services Transformation 
project intended to address these and other areas, it 
will take several years before many of the issues we 
identified can be effectively addressed. Among our 
more significant findings were the following: 



• In half the cases we reviewed, agencies lacked 
supporting documentation to adequately 
demonstrate a person's eligibility or needs. 
As a result, agencies could not demonstrate, 
and the Ministry could not assess, whether 
the individual was receiving the appropriate 
level of service or was in need of additional 
support. 

• The Ministry has not established accept- 
able standards of service, or the necessary 
processes to properly monitor the quality of 
services provided. Consequently, it cannot 
assess whether it is receiving value for money 
for the funding provided to community-based 
agencies. Ministry staff rarely visit agencies 
for these purposes. 

• The Ministry is not aware of the number of 
people who are waiting for agency-based sup- 
portive services, information that is necessary 
for assessing unmet service needs. 

• Although one would expect a consistent set 
of rules about what are appropriate services 
and, therefore, allowable expenditures under 
the Passport program, the Ministry has not set 
such rules. As a result, expenses for services 
reimbursed in one region were deemed ineli- 
gible for reimbursement in another. 

• In practice, annual agency funding continues 
to be primarily historically based rather than 
needs-based. This exacerbates any previous 
funding inequities. As a result, some hourly 
service costs appeared excessive, and the 
range of costs per hour for similar services 
varied widely across the province. 

• The Ministry had little knowledge of whether 
the agencies it funded and their boards of 
directors had effective governance and control 
structures in place. 

• As of March 31, 2011, there was a waiting list 
of almost 9,600 people who met the SSAH eli- 
gibility criteria but were still waiting for SSAH 
funding. 



Chapter 2 



Public Accounts 
of the Province 



Introduction 



Ontario's Public Accounts for each fiscal year end- 
ing on March 31 are prepared under the direction 
of the Minister of Finance, as required by the Finan- 
cial Administration Act (Act). The Public Accounts 
comprise the province's annual report, including 
the province's consolidated financial statements, 
and three supplementary volumes of additional 
financial information. 

The government's responsibility for preparing 
the consolidated financial statements encompasses 
ensuring that the information, including the many 
amounts based on estimates and judgment, is pre- 
sented fairly. The government is also responsible for 
ensuring that a system of control, with supporting 
procedures, is in place to provide assurance that 
transactions are authorized, assets are safeguarded, 
and proper records are maintained. 

My Office audits these consolidated financial 
statements. The objective of our audit is to obtain 
reasonable assurance that the statements are free of 
material misstatement — that is, free of significant 
errors or omissions. The consolidated financial 
statements, along with my Independent Auditor's 
Report on them, are included in the province's 
annual report. 

The province's 2010/11 annual report also con- 
tains a Financial Statement Discussion and Analysis 
section that provides additional information about 



the province's financial condition and fiscal results 
for the year ended March 31, 2011, including some 
details of what the government accomplished in 
the 2010/11 fiscal year. The provision of such infor- 
mation enhances the fiscal accountability of the 
government to both the Legislative Assembly and 
the public. 

The three supplementary volumes of the Public 
Accounts consist of the following: 

• Volume 1 — statements from all ministries and 
a number of schedules providing details of the 
province's revenues and expenses, its debts 
and other liabilities, its loans and investments, 
and other financial information; 

• Volume 2 — audited financial statements of 
significant provincial corporations, boards, 
and commissions whose activities are 
included in the province's consolidated finan- 
cial statements, as well as other miscellaneous 
financial statements; and 

• Volume 3 — detailed schedules of ministry 
payments to vendors and transfer-payment 
recipients. 

My Office reviews the information in the prov- 
ince's annual report and in Volumes 1 and 2 of the 
Public Accounts for consistency with the informa- 
tion presented in the province's consolidated finan- 
cial statements. 

The Act requires that, except in extraordinary 
circumstances, the government deliver its annual 
report to the Lieutenant Governor in Council within 
180 days of the end of the fiscal year. The three 
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supplementary volumes must be submitted to the 
Lieutenant Governor in Council within 240 days 
of the end of the fiscal year. Upon receiving these 
documents, the Lieutenant Governor in Council 
must lay them before the Legislative Assembly or, 
if the Assembly is not in session, make the informa- 
tion public and then lay it before the Assembly 
within 10 days of the time it resumes sitting. 

This year, the government released the prov- 
ince's 2010/11 Annual Report and Consolidated 
Financial Statements, along with the three Public 
Accounts supplementary volumes, on August 23, 
2011, meeting the 180-day deadline. 

In conducting our annual audit of the Public 
Accounts we work closely with the Ministry of 
Finance and particularly with the Office of the 
Provincial Controller. While we may not always see 
eye-to-eye on all issues, our working relationship 
has always been professional and constructive. 

mm 



Consolidated Financial 
Statements 



The Auditor General Act requires that I report 
annually on the results of my examination of the 
province's consolidated financial statements. I am 
pleased to report that my Independent Auditor's 
Report to the Legislative Assembly on the province's 
consolidated financial statements for the year 
ended on March 31, 2011, is clear of any qualifica- 
tions and reservations, and reads as follows: 

Independent Auditor's Report 

To the Legislative Assembly of the Province 
of Ontario 

I have audited the accompanying consoli- 
dated financial statements of the Province 
of Ontario, which comprise the consoli- 
dated statement of financial position as at 



March 31, 2011, and the consolidated state- 
ments of operations, change in net debt, 
change in accumulated deficit, and cash 
flow for the year then ended and a summary 
of significant accounting policies and other 
explanatory information. 

Management's Responsibility for the Consoli- 
dated Financial Statements 
The Government of Ontario is responsible 
for the preparation and fair presentation 
of these consolidated financial statements 
in accordance with Canadian public sector 
accounting standards, and for such internal 
control as the Government determines is 
necessary to enable the preparation of con- 
solidated financial statements that are free 
from material misstatement, whether due to 
fraud or error. 

Auditor's Responsibility 
My responsibility is to express an opinion 
on these consolidated financial statements 
based on my audit. I conducted my audit 
in accordance with Canadian generally 
accepted auditing standards. Those stan- 
dards require that I comply with ethical 
requirements and plan and perform the 
audit to obtain reasonable assurance about 
whether the consolidated financial state- 
ments are free from material misstatement. 

An audit involves performing procedures to 
obtain audit evidence about the amounts 
and disclosures in the consolidated financial 
statements. The procedures selected depend 
on the auditor's judgment, including the 
assessment of the risks of material misstate- 
ment of the consolidated financial state- 
ments, whether due to fraud or error. In 
making those risk assessments, the auditor 
considers internal control relevant to the 
entity's preparation and fair presentation 
of the consolidated financial statements in 
order to design audit procedures that are 



Public Accounts of the Province 

the applicable accounting standards, and its 
responsibility for internal control to enable 
the preparation of financial statements free 
from material misstatement, whether due to 
fraud or error; and 
• Paragraph on Auditor's responsibility — the 
standards now require a clearer description of 
the Auditor's responsibility with respect to the 
audit, including a more detailed description 
on the process for performing the audit, that 
the auditor needs sufficient appropriate audit 
evidence, and that the Auditor is expressing 
no opinion on the effectiveness of internal 
control. 
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appropriate in the circumstances, but not 
for the purpose of expressing an opinion 
on the effectiveness of the entity's internal 
control. An audit also includes evaluating 
the appropriateness of accounting policies 
used and the reasonableness of accounting 
estimates made by the Government, as well 
as evaluating the overall presentation of the 
consolidated financial statements. 

I believe that the audit evidence I have 
obtained is sufficient and appropriate to 
provide a basis for my opinion. 

Opinion 

In my opinion, these consolidated financial 
statements present fairly, in all material 
respects, the consolidated financial position 
of the Province of Ontario as at March 31, 
2011 and the consolidated results of its 
operations, change in its net debt, change 
in its accumulated deficit, and its cash 
flows for the year then ended in accordance 
with Canadian public sector accounting 
standards. 

[signed] 

Jim McCarter, FCA 
Toronto, Ontario Auditor General 
August 8, 2011 Licensed Public Accountant 

As a result of new Canadian Auditing Standards 
that came into effect in December 2010, my report 
this year differs from those of previous years. The 
following are some of the key changes in this year's 
report that reflect the requirements of the new 
standards. 

• Title — the title of the report now includes 
"independent" to clearly convey to readers the 
auditor's independence; 

• Paragraph on Management's responsibility — 
the report now includes a separate paragraph 
that describes the government's responsibility 
for the preparation and fair presentation of 
the financial statements in accordance with 



Government debt has become a worldwide eco- 
nomic issue, especially since the global recession 
of 2008 that saw governments around the world 
incur large deficits to pay for stimulus programs to 
combat the effects of the recession. 

Ontario has not been immune to the recession, 
with the government reporting large deficits in 
each of the last three fiscal years. The government 
is also projecting shortfalls for the next six years, as 
illustrated in Figure 1. 

The government's projections indicate that by 
the time Ontario revenues are sufficient to meet its 
expenses in the 2017/18 fiscal year, the combined 
annual deficits from 2008/09 to 2016/17 will total 
almost $110 billion. The government will need to 
issue new debt to finance these projected annual 
deficits, which will increase Ontario's current debt 
load significantly before it is able to balance its 
books in six years' time. 

In this section, we first highlight the different 
measures of government debt — that is, the differ- 
ent ways government debt can be looked at. We 
then compare the province's growing debt to the 
strength of the provincial economy, and to the 
debt burden of other governments for perspective. 
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Figure 1: Ontario Revenue and Expenses, 2008/09-2017/18 ($ billion) 

Source of data: 2010/11 Province of Ontario Consolidated Financial Statements and 2011 Ontario Budget 
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Finally, we highlight some of the consequences to 
the province as a result of carrying a significant 
debt load. 

DIFFERENT MEASURES OF DEBT 

The government's debt can be measured in a 
number of ways. Figure 2 provides details on the 
debt over the last four fiscal years as reported in the 
province's consolidated financial statements, along 
with projections over the next three fiscal years as 
reported in the 2011 Budget. 

Definitions of the province's three measures of 
debt in Figure 2 are as follows: 

• Total debt represents the total amount of 
money the government owes to outsiders 
and consists of bonds issued in public capital 
markets, non-public debt, T-bills, and U.S. 
commercial paper. It provides the broadest 
measure of a government's debt load and its 
total borrowings to date. 

• Net debt is the difference between the gov- 
ernment's total liabilities and its financial 
assets. Liabilities consist of all amounts the 
government owes to external parties, includ- 
ing total debt, accounts payable, pension and 
retirement obligations, and transfer payment 
obligations. Financial assets are those that can 
be used to pay off liabilities or finance future 
operations, and include cash, accounts receiv- 
able, temporary investments, and investments 
in government business enterprises. Net debt 



provides a measure of the amount of future 
revenues required to pay for past government 
transactions and events. 
• Accumulated deficit represents the sum of all 
past government annual deficits and surpluses. 
It is derived by taking net debt and deducting 
the value of the government's non-financial 
assets, such as its tangible capital assets. 
Net debt is generally considered to be a use- 
ful indicator of a government's financial position 
and one that provides insight into how well a 
government can afford to provide future services. 
A significant net-debt position reduces the ability 
of a government to devote financial resources and 
future revenues to the provision of public services. 
The Canadian Institute of Chartered Accountants 
publication entitled 20 Questions about Government 
Financial Reporting notes that net debt is an import- 
ant measure of a government's fiscal capacity. 

The government, on the other hand, considers 
the accumulated deficit to be a better measure for 
evaluating its financial position and its capacity to 
deliver future services, because the accumulated 
deficit takes into account the acquisition of non- 
financial assets, such as tangible capital assets, 
using debt. Under the Fiscal Transparency and 
Accountability Act, 2004 (FTAA) the government is 
required to maintain a prudent ratio of provincial 
debt (defined in the FTAA as the accumulated defi- 
cit) to Ontario's gross domestic product, which is 
discussed in more detail in the next section. 
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Figure 2: Total Debt, Net Debt, and Accumulated Deficit, 2007/08-2013/14 ($ million) 

Source of data: 2010/11 Province of Ontario Consolidated Financial Statements, 2011 Ontario Budget, and Office of the Auditor General of Ontario 
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MAIN CONTRIBUTORS TO NET-DEBT 
GROWTH 

For the most part, the province's growing net debt 
since the 2007/08 fiscal year is attributable to large 
deficits in recent years, along with investments in 
capital assets such as buildings, other infrastructure, 
and machinery and equipment acquired directly by 
the government or its consolidated organizations, 
such as public hospitals, as illustrated in Figure 3. 

While the government has not provided details 
on its debt beyond the 2013/14 fiscal year, we 
estimate that Ontario's net debt could surpass 
$300 billion by the 2017/18 fiscal year, based on the 
government's deficit projections in the 2011 Budget 
and assuming government investments in capital 
assets continue at the levels of recent years. 

In summary, Ontario's net debt will increase 
from $157 billion at the end of the 2007/08 fis- 
cal year to over $300 billion by 2017/18, in effect 
almost doubling in the 10-year period before the 



government projects it will be able to bring its books 
back into balance. Accordingly, the amount of debt 
owed by each resident of Ontario on behalf of the 
government will increase from about $12,000 per 
person in 2008 to about $21,000 per person in 2018. 

ONTARIO'S RATIO OF NET DEBT TO GDP 

The level of debt relative to the size of the econ- 
omy — the ratio of debt to the market value of all 
goods and services produced over a defined per- 
iod, called the gross domestic product (GDP) — is 
generally considered to be a good indicator of a 
government's ability to manage its debt load. The 
ratio of net debt to GDP measures the relationship 
between a government's obligations and its capacity 
to raise funds to meet them. When the ratio is rising, 
it means that the government's net debt is growing 
at a faster rate than the provincial economy. 



Figure 3: Net Debt Growth Factors, 2007/08-2013/14 ($ million) 

Source of data: 2010/11 Province of Ontario Consolidated Financial Statements, 2011 Ontario Budget, and Office of the Auditor General of Ontario 
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1. Includes investments in government-owned land, buildings, machinery and equipment, and infrastructure assets capitalized during the year less annual 
amortization and net gains reported on sale of government-owned tangible capital assets. 

2. Unrealized Fair Value Losses/(Gains) on the Ontario Nuclear Funds Agreement (ONFA) Funds held by Ontario Power Generation Inc. 
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The province's net debt-to-GDP ratio was rela- 
tively stable between the 2005/06 and 2007/08 
fiscal years, averaging about 30% at each fiscal 
year-end, as illustrated in Figure 4. The ratio then 
began to increase in 2008/09 and is projected to 
peak at 40% in 2014/15 before starting to fall. 

The net debt-to-GDP ratio shows that govern- 
ment debt will grow at a faster rate than the prov- 
incial economy until the 2014/15 fiscal year and 
will begin to fall only in 2015/16, when the rate of 
expected government debt growth will fall below 
the expected growth rate of the provincial economy 

Another useful exercise in assessing Ontario's 
debt load is to compare it with other Canadian 
jurisdictions. The net debt of most provinces and 
the federal government, along with their respective 
ratios of net debt to GDP, is illustrated in Figure 5. 

Generally, the western provinces have a signifi- 
cantly lower net debt-to-GDP ratio than Ontario, 
while the Maritime provinces are roughly similar to 
Ontario and Quebec has a significantly higher ratio 
than Ontario. 

Ontario's net debt-to-GDP ratio is lower than 
that of the United States and several European 
countries that also carry significant debt loads, as 
illustrated in Figure 6. Although caution is war- 
ranted because there may be differences in how 

Figure 4: Ontario Net Debt-to-GDP Ratio, 2005/06- 
2017/18 (%) 

Source of data: 2010/11 Province of Ontario Annual Report and Consolidat- 
ed Financial Statements 
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Figure 5: Net Debt and the Net Debt-to-GDP Ratios of 
Canadian Jurisdictions, 2010/11 

Source of data: 2010/11 Province of Ontario Annual Report and 
Consolidated Financial Statements, 2011 Federal Budget, budget updates 
and 2011 budgets of selected provincial jurisdictions, and Office of the Aud- 
itor General of Ontario 
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these countries define liabilities and financial assets 
relative to Ontario, the information does provide a 
useful level of comparison. 

CONSEQUENCES OF HIGH 
INDEBTEDNESS 

As any householder knows, there are consequences 
to high levels of indebtedness. The same applies to 
governments, including the following: 

• Debt Costs Take Funding Away from Other 
Government Programs — As provincial indebt- 
edness grows, so does the cash needed to pay 
the interest costs to service the debt. Higher 
interest costs consume a greater propor- 
tion of government resources, limiting the 
amount the government can spend on other 
things. To put this "crowding out" effect into 
perspective, the government currently spends 
more on debt interest than it does on post- 
secondary education, and these interest costs 
are growing. 

The government's debt-servicing cost in 
the 2008/09 fiscal year was $8.5 billion and 
rose to $9.5 billion in 2010/11. It is projected 
to rise to $16.3 billion by the time the province 
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Figure 6: Comparison of Net Debt-to-GDP Ratio of 
Ontario and Selected Industrialized Countries, 
September 2011 (%) 

Source of data: 2010/11 Province of Ontario Annual Report and Consolidated 
Financial Statements, and Office of the Auditor General of Ontario 
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balances its books in 2017/18. As a percentage 
of total provincial revenues, debt-servicing 
cost will rise from 8.9% in 2010/11 to an 
estimated 11.5% by 2017/18. In other words, 
by 2017/18, it is estimated that 11.5 cents of 
every taxpayer dollar will go towards paying 
only the annual interest on the debt. 

• Higher Sensitivity to a Rise in Interest Rates — 
Over the last few years, governments gener- 
ally have been able to use low interest rates 
to finance higher debt loads. For example, 
Ontario was paying an average effective inter- 
est rate of about 8% in 2000, but that had 
dropped to less than 5% in 2011. However, 
higher debt levels increase the province's 
sensitivity to any rate increases. For example, 
in its 2011 Ontario budget, the government 
noted that, at its current debt level, a 1% 
increase in rates would add an additional 
$500 million to its interest costs. 

• Credit Ratings and Investor Sentiment — 
Credit-rating agencies assess a government's 
creditworthiness largely based on its capacity 
to manage its debt, and they consider such 
factors as that government's economic resour- 
ces, institutional strengths, financial health, 



and susceptibility to major risks. This rating 
has an impact on the cost of future govern- 
ment borrowings because, generally speaking, 
a lower credit rating means investors will 
demand a greater risk premium in the form of 
higher interest rates before they are willing to 
purchase that jurisdiction's debt. 

With respect to Ontario's credit rating, 
one TD Canada Trust banking analyst noted 
in March 2011 that "there is little evidence 
that bond investors are getting nervous about 
Ontario's fiscal situation or that rating agen- 
cies will be downgrading Ontario from its 
longstanding AA-rating." However, analysts 
also warn that the government's large bor- 
rowing requirements, along with its increas- 
ing reliance on foreign investors, does raise 
the risk of a credit-rating downgrade. Any 
such change in the credit rating would force 
Ontario to pay higher interest on its future 
borrowings. 

CONCLUSION 

A government's debt has been described as a 
burden placed on future generations, especially 
debt used to finance operating deficits. Debt used 
to finance infrastructure investments is more likely 
to leave behind investments that future generations 
can benefit from. 

It is important to note that while the govern- 
ment has presented a plan to eliminate the annual 
deficit by 2017/18, no clear strategy or forecast has 
been articulated for paying down its existing and 
future debt. 

Once annual deficits are no longer the norm, 
one strategy for paying down debt is to hold the line 
on any future debt increases and use the additional 
revenues generated by a growing economy to start 
to reduce the debt. In any case, regardless of what 
strategy is contemplated, we believe the govern- 
ment should provide legislators and the public with 
long-term targets and a strategy for how it plans to 
address the current and projected debt burden. 
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Update on the Workplace 
Safety and Insurance Board 



In our 2009 Annual Report, we suggested that the 
government reconsider the exclusion from the 
province's consolidated financial statements of 
the financial results of the Workplace Safety and 
Insurance Board (WSIB). The exclusion of the 
WSIB's financial results is based on its classification 
by the government as a "trust." However, given its 
significant unfunded liability and other factors, we 
questioned whether the WSIB was operating like 
a true trust for financial-statement purposes as 
prescribed by accounting standards of the Public 
Sector Accounting Board. 

The WSIB's unfunded liability as of Decem- 
ber 31, 2008, totalled $11.5 billion. It had grown 
to $11.8 billion as of December 31, 2009, and to 
$12.4 billion by December 31, 2010. This compares 
to an unfunded liability of $5.9 billion in 2006. 
If the WSIB had been included in the govern- 
ment reporting entity for the 2010/11 fiscal year, 
Ontario's deficit would have been approximately 
$330 million higher than reported, and the prov- 
ince's net debt and accumulated deficit would 
have increased by almost 5% and more than 7%, 
respectively. Clearly, the inclusion of the WSIB in 
the province's financial statements would have a 
material impact on the province's consolidated 
financial statements. 

In Chapter 4 of this Annual Report, we follow up 
our 2009 review of the WSIB's unfunded liability 
and provide an update on actions taken by the WSIB 
and the government following our 2009 review. 
According to information we received from the 
WSIB and the Ministry of Labour, and discussions 
we had with senior WSIB management, a number 
of initiatives are under way to address its unfunded- 
liability situation. For instance, the WSIB launched 
an independent funding review, led by an external 
academic, that is seeking advice from the various 
stakeholders on how best to address the unfunded- 



liability situation. As well, legislation has been 
passed that, subject to proclamation, would require 
that the WSIB reach a prescribed level of funding 
within a specified time frame. The funding and time 
frame are to be established by regulation that will 
take the results of the current independent funding 
review into consideration. 

As a result of these initiatives to address WSIB's 
unfunded liability, we agree for the time being 
with the government that the WSIB can retain its 
"trust" status. However, we will continue to monitor 
the progress being made toward addressing the 
significant unfunded liability and, if we believe it is 
insufficient, we will re-evaluate our position. 
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Update on the Pensio 
Benefit Guarantee Fun 



The Pension Benefit Guarantee Fund (PBGF) 
guarantees the payment of certain pension benefits 
when eligible private-sector defined-benefit plans 
are terminated under conditions specified in the 
Pension Benefits Act (Act). Under the Act, the PBGF 
is funded through premiums paid by private-sector 
pension plan sponsors. Participation in the PBGF is 
mandatory for many defined-benefit plans registered 
in Ontario. The PBGF is intended to be self-financing, 
with funding in the form of annual payments based 
on per-member and risk-related fees. 

As with the WSIB, the PBGF is classified as a 
trust in the province's consolidated financial state- 
ments. This means its assets, liabilities, and operat- 
ing results are excluded from the accounts of the 
province, but must be disclosed in the notes to the 
province's consolidated financial statements. 

Recent corporate insolvencies and bankruptcies 
arising from the economic downturn and other 
factors have led to greater claims on the PBGF. As 
a result, the PBGF reported unfunded liabilities of 
$102 million as of March 31, 2008, and $47 million 
as of March 31, 2009, as shown in Figure 7. These 
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Figure 7: PBGF Financial Position, 2007/08-2010/11 ($ million) 

Source of data: PBGF 
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1. Includes a $500-million grant from the province 

2. Most relate to claims for pension payments on terminated pension plans 

unfunded liabilities existed despite a $330-million 
interest-free loan from the province in the 2003/04 
fiscal year, to be repaid in $ll-million annual 
installments over 30 years. 

In 2009, the government amended the Pen- 
sion Benefits Act to clarify that the PBGF is a self- 
sustaining fund, independent of the government. 
The amendments allow, but do not require, the 
government to provide grants or loans to the PBGF. 
The amended act also emphasized that the PBGF's 
liabilities are limited to its assets. 

On March 25, 2010, the Legislative Assembly 
approved a $500-million grant to the PBGF to 
help stabilize its financial position and cover the 
costs of recent plan windups. As a result of this 
grant, the PBGF reported a surplus of $103 mil- 
lion as of March 31, 2010. As of March 31, 2011, 
notwithstanding the previous year's $500-million 
government cash infusion, it reported an unfunded 
liability of $6 million as a result of expenses exceed- 
ing revenues by $109 million. Therefore, the gov- 
ernment's previous infusion has already been fully 
depleted. 

An independent actuary appointed by the gov- 
ernment to review the stability and the financial 
status of the PBGF noted in June 2010 that in the 
absence of an increase in private-sector member 
assessments, the Fund would require an upfront 
reserve (net of current claims as of January 2010) 
of between $680 million and $1,023 billion to cover 
expected future claims. The actuary determined 
that in order to be considered self-sufficient over 



the long term and cover existing loan repayments 
and expected future claims plus expenses, the PBGF 
would require a 450% increase in the employer- 
and employee-assessment rates to fund benefits at 
the current maximum coverage level of $1,000 per 
month per employee. 

In August 2010, the government also announced 
a four-part strategy to further mitigate risks and 
enhance the sustainability of the PBGF as follows: 

• build reserves through the $500 million grant, 
provided in March 2010; 

• raise future PBGF revenues by increasing 
assessments in 2012; 

• reduce the level of risk to the PBGF by 
extending the eligibility period for covering 
new plans and benefit improvements from 
three to five years; and 

• reduce the PBGF's exposure by strengthening 
pension funding rules. 

While we acknowledge that the government has 
taken steps to attempt to place the PBGF on a more 
stable financial footing, we believe that the PBGF 
did not meet the criteria to retain its "trust" classi- 
fication for the 2010/11 fiscal year, given its history 
of requiring government funding to meet its finan- 
cial obligations and the actuary's suggestion that 
this dependency will likely continue in the future. 
In our opinion, if the government chooses to step 
in periodically to provide financial resources to this 
organization, then it does not meet the definition of 
a "trust," nor the intent of the accounting standard 
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that allows standalone trusts to be excluded from a 
government's financial statements. 

However, this year we concluded the impact of 
excluding the PBGF from the government's con- 
solidated financial statements was not significant 
enough to affect our March 31, 2011, audit opinion. 
We will continue to recommend that the Ministry of 
Finance include the PBGF in the province's consoli- 
dated statements until such time as there is signifi- 
cant improvement, without government assistance, 
in the financial position of the PBGF. 



MINISTRY RESPONSE 



The Ministry appreciates the Auditor General's 
recognition of the government's four-part 
strategy to strengthen the sustainability of the 
PBGF, including measures to increase the fund's 
revenues and limit its liabilities. 

In the Ministry's view, these proposals 
establish a stable financial basis for the fund, 
permitting it to maintain its trust status and 
eliminating the need for consolidation. 



Review of the 2011 
Pre-Election Report on 
Ontario's Finances 



The Fiscal Transparency and Accountability Act, 
2004 (Act) requires the Minister of Finance to 
issue a report on Ontario's finances in advance of 
a provincial election. The purpose of this report is 
to provide the public with detailed information on 
the province's estimated future revenues, expenses, 
and projected surplus or deficit for the next three 
fiscal years. The Act requires the Auditor General to 
review the government's report to determine if it is 
reasonable, and to release a report describing the 
results of this review. 

As a provincial general election had been called 
for October 6, 2011, the government released its 



2011 Pre-Election Report on Ontario's Finances on 
April 26, 2011. The fiscal plan on which the pre- 
election report was based was set out in the 2011 
Ontario Budget. 

As required by the Act, the report provided 
information on: 

• the macroeconomic forecasts and assumptions 
used to prepare the government's fiscal plan; 

• an estimate of Ontario's revenues and 
expenses, including estimates of the major 
components of the revenues and expenses; 

• details about the budget reserve required to 
provide for unexpected adverse changes in 
revenues and expenses; and 

• the ratio of provincial debt to Ontario's gross 
domestic product. 

A key principle of the Act is that Ontario's 
fiscal policy be based on cautious assumptions. 
Overall, we concluded that the government based 
its estimates of revenues and interest on the public 
debt on prudent and cautious assumptions. How- 
ever, we concluded that many of the assumptions 
underlying its estimates for program expenses 
(that is, expenses excluding interest on the public 
debt and reserves) were optimistic and aggressive 
rather than cautious. This was especially the case 
for public-sector salaries and for health-care costs, 
which together account for the majority of program 
expenses. 

We cautioned that since the pre-election report 
is essentially a forecast, actual results will undoubt- 
edly differ from its estimates. Given that many of 
the assumptions underlying the expense projec- 
tions are optimistic rather than cautious, there is a 
heightened risk that actual expenses will be higher 
than estimated. Unless revenue growth is higher 
than expected to compensate for higher expenses, 
annual deficits may also turn out to be higher 
than planned. In that case, the government will 
need to consider additional changes in policy or 
operations to achieve the fiscal targets set out in 
the 2011 Budget. 
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Future Public Accounts 
Issues 



THE IMPORTANCE OF PUBLIC-SECTOR 
ACCOUNTING STANDARDS 

In Chapter 2 of my 2008 and 2010 Annual Reports, I 
discussed the importance of governments adhering 
to generally accepted accounting standards estab- 
lished by an independent standard-setting body in 
order to produce credible financial statements and 
information on which the public can rely 

Accounting standards specify when transactions 
are to be recognized and how they are to be meas- 
ured and disclosed in financial statements. In order 
to be authoritative, accounting standards should be 
established by a recognized professional standard- 
setting body through an organized, open, and 
transparent public process. 

In Canada, the Public Sector Accounting Board 
(PSAB) of the Canadian Institute of Chartered 
Accountants (CICA) is the authoritative body that 
establishes accounting standards for the public sec- 
tor. PSAB standards represent generally accepted 
accounting principles for governments in Canada 
and are a primary source of guidance for public- 
sector accounting. 

PSAB emphasizes "due process" in setting its 
standards in order to maintain objectivity and 
ensure that the views of all interested parties are 
heard and considered. In developing or revising an 
accounting standard, PSAB typically follows a five- 
step process: 

• basic research; 

• approval of a project proposal; 

• issuing a statement of principles to a 
designated group of accountants and non- 
accountants for initial feedback; 

• issuing one or more public exposure drafts 
and soliciting comments from all interested 
individuals or organizations; and 

• approving and publishing a final standard. 



PSAB has been under significant pressure 
recently from certain stakeholders. Some govern- 
ments, for instance, have expressed concerns 
that PSAB standards do not adequately take into 
account the unique challenges facing governments 
when they make decisions on financial reporting, 
budgeting, and fiscal policy. While PSAB must 
ensure that new accounting standards take all of 
these concerns into consideration, it is also con- 
strained by the need to ensure such standards are 
consistent with its conceptual framework. 

PSAB's conceptual framework consists of a set 
of overarching and interrelated objectives, funda- 
mental principles, and definitions that establish 
how assets, liabilities, revenues, and expenses arise, 
and how they are to be measured and disclosed. 
The conceptual framework was designed to help 
develop accounting standards that will consistently 
produce financial statements that most fairly reflect 
the results of an entity's operations and its financial 
position at the end of a reporting period. Account- 
ing-standard-setting bodies around the world use 
such conceptual frameworks to ensure that any 
proposed accounting standards are theoretically 
sound. PSAB is currently undertaking a review of its 
conceptual framework, which is discussed later in 
this chapter. 

RECENT PSAB ACHIEVEMENTS 

PSAB resolved a number of significant financial 
accounting and reporting issues in the 2010/11 
fiscal year. These include Government Transfers, 
addressed in new standard PS3410, and Financial 
Instruments, addressed in new standard PS3450. 
Both are discussed below. 

One key message we want to convey to readers 
through this discussion is that the public inter- 
est has been well served by PSAB's role in setting 
independent and conceptually based accounting 
standards for governments. Since its establishment 
in 1981 and its gradual acceptance by Canadian 
governments as the authoritative issuer of public- 
sector accounting standards, PSAB has contributed 
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enormously to improving the completeness, rel- 
evance, reliability, and comparability of Canadian 
government financial statements. 

We want to acknowledge that the province's 
consolidated financial statements, in all material 
respects, have consistently complied with PSAB 
standards and that all governments over the years 
have been diligent in their efforts to improve the 
clarity and completeness of their statements and 
annual reports. We believe it is critical that Ontario 
continue to prepare its financial statements in 
accordance with PSAB standards so that legislators 
and the public can rely on the reported annual sur- 
plus or deficit as being a fair reflection of what has 
actually transpired with respect to the government's 
management of the public purse for the year. 

RATE-REGULATED ASSETS AND 
LIABILITIES 

Current Situation 

Over the last three years, we have raised concerns 
about the appropriateness of recognizing rate- 
regulated assets and liabilities in the government's 
consolidated financial statements. Rate-regulated 
accounting practices were developed to recognize 
the unique nature of regulated entities such as elec- 
tricity generators, transmitters, and distributors. 
Under rate-regulated accounting, a regulator estab- 
lished under legislation such as the Ontario Energy 
Board approves the prices that a regulated entity 
can charge customers and often allows regulated 
entities to defer (record as an asset) certain costs 
for recovery in future periods that, under normal 
generally accepted accounting principles (GAAP), 
would be expensed in the year incurred. 

In Ontario, there are three major provincially 
owned organizations in the electricity sector — 
Ontario Power Generation Inc. (OPG), Hydro One 
Inc. (Hydro One), and the Ontario Power Authority 
(OPA) — that use rate-regulated accounting. The 
financial position and operating results of these 
three organizations are included in the govern- 



ment's consolidated financial statements. The net 
effect of including the impact of rate-regulated 
accounting in the 2010/11 fiscal year was to 
decrease the reported deficit by $23 million. While 
this year's deficit impact was quite small, the 
impact can also be quite large, as was the case in 
the 2009/10 fiscal year, when its net effect was to 
reduce the reported deficit by $1.1 billion. 

Up to now and including the 2010/11 fiscal year, 
the use of rate-regulated accounting by certain 
rate-regulated entities was allowed under Canadian 
GAAP. Specifically, PSAB's accounting standards 
allowed OPG and Hydro One, which are defined 
as government business enterprises, to be con- 
solidated without their accounting policies being 
adjusted to remove the impact of rate-regulated 
accounting. Given PSAB's position, we accepted this 
accounting treatment. However, from a theoretical 
viewpoint, we continued to question whether rate- 
regulated assets and liabilities met the definition of 
bona fide assets or liabilities for the purposes of the 
government's consolidated financial statements. In 
the case of the OPA, which does not meet the PSAB 
criteria of being a government business enterprise, 
the impact of rate-regulated accounting on the 
OPA's results should have been removed before the 
OPA was included in the consolidated statements. 
In this case, not making the adjustment did not have 
a material effect on the province's reported results 
and therefore did not affect our audit opinion. 

Looking Forward 

The era of rate-regulated accounting appears to 
be coming to a close, at least for jurisdictions such 
as Canada that are converting to international 
accounting standards. Last year, both the Inter- 
national Accounting Standards Board (IASB) and 
the CICA's Accounting Standards Board issued 
exposure drafts that, if approved, would have 
allowed rate-regulated entities to continue recog- 
nizing regulatory assets and liabilities under certain 
conditions. However, while the recommendations 
of these exposure drafts were overwhelmingly 
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supported by the utility industry, the majority 
of accounting bodies and standards-setters who 
responded disagreed with the recommendations. 
Accordingly, the IASB has deferred the current pro- 
ject, and it is unclear if and when any future project 
on rate-regulated accounting will be initiated. 
The CICA's Accounting Standards Board recently 
indicated that it would not consider any "local" 
amendments to International Financial Reporting 
Standards (IFRS) to allow for rate-regulated 
accounting in Canada. Instead, the treatment of all 
assets and liabilities in future will have to follow 
normal generally accepted accounting principles, 
and rate-regulated assets and liabilities will no 
longer be considered acceptable. 

The accounting standard-setter in the United 
States, the Financial Accounting Standards Board, 
is not going along with the views of its international 
counterparts. For now, it has decided not to adopt 
IFRS, and it will continue to allow rate-regulated 
accounting. 

Ontario's Ministry of Finance contends that the 
province's rate-regulated assets and liabilities meet 
PSAB's standards without reference to any of the 
rate-regulated provisions in the CICA Handbook. 
As the Ministry is aware, we do not agree with this 
position. 

In its March 31, 2011 Annual Report and Con- 
solidated Financial Statements, the government spe- 
cifically commented on this issue. The note entitled 
Future Changes to Accounting Standards stated: 

At present, IFRS does not address rate- 
regulated accounting and it is uncertain 
if or when such standards might be intro- 
duced by the IASB. The government plans 
to provide direction to certain controlled 
rate-regulated entities to ensure that the 
financial reports of these entities follow 
accounting standards that it believes 
best represent the economic substance of 
transactions and best serve the informa- 
tion needs to different users. 



We noted that the government has recently 
directed Hydro One to prepare its future financial 
statements in accordance with U.S. GAAP through 
the passage of regulation 395/11 under the Finan- 
cial Administration Act. We want to reiterate that 
it is not the impact of this decision on Hydro One's 
financial statements that is our direct concern — 
rather, we are concerned about what effect these 
developments may have on the province's consoli- 
dated financial statements. 

As the auditors of the province's consolidated 
financial statements, we have concerns about 
Hydro One's use of U.S. GAAP and about the 
future disposition of the rate-regulated assets and 
liabilities of OPG and the OPA. The province uses 
Canadian GAAP in preparing its statements. If Can- 
adian GAAP does not allow rate-regulated assets 
and liabilities to be recorded, there may be an issue 
in next year's audit if such assets and liabilities are 
nevertheless included in the province's consoli- 
dated financial statements due to the consolidation 
of OPG, Hydro One, and the OPA in those results, 
and if that inclusion has a material impact. 

We are also concerned about the province using 
legislation to override Canadian GAAP — a theme 
that we have raised in our last two Annual Reports. 
This year, the government passed a regulation 
requiring Hydro One to use U.S. GAAP to allow it 
to continue to include the impact of rate-regulated 
activities in its future financial statements. This 
represents a departure from the historical tradition 
in Ontario of complying with Canadian accounting 
standards. 



| MINISTRY RE: 




The Ministry recognizes the challenges in the 
accounting-standards-setting environment to 
achieve consensus on the required approach for 
rate-regulated accounting. Given the deferral 
by the standards-setters to resolve this issue, 
the government directed Hydro One to follow 
U.S. GAAP to allow it to continue to account for 
these rate-regulated assets and liabilities as it 
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has historically been able to do under Canadian 
GAAP. This decision is consistent with actions 
by both the Canadian Securities Administrators 
and the Ontario Securities Commission that 
have enabled rate-regulated utilities to submit 
their financial statements on a U.S. GAAP basis 
until 2014. 

The government looks forward to the 
standards-setters undertaking actions to com- 
plete their efforts to resolve the outstanding 
rate-regulated accounting concerns and update 
the standards accordingly. 

ACCOUNTING FOR GOVERNMENT 
TRANSFERS 

The Government Transfers project was initiated 
by PSAB a number of years ago to address several 
accounting issues related to monetary transfers 
from one level of government to another, including: 

• accounting appropriately for multi-year fund- 
ing provided by one government to another; 

• clarifying the authorization needed for trans- 
fers to be recognized; 

• clarifying the degree to which stipulations 
imposed by a transferring government affect 
the timing of transfer recognition in the 
accounts of both the transferring and recipi- 
ent governments; and 

• appropriately accounting for transfers that 
are to be used to acquire or construct tangible 
capital assets. 

One of the most difficult areas to address was 
how recipients should account for multi-year trans- 
fers. For instance, if the federal government made a 
lump-sum transfer near the end of the fiscal year to 
a province that was to be used to fund services over 
several years, should the province immediately rec- 
ognize the full grant as revenue or should the grant 
be taken into revenue only as it is being spent on the 
services for which it was provided? A similar issue 
arose with respect to capital transfers. A number of 



stakeholders took the view that a capital transfer 
should be recognized as revenue when the recipient 
government incurred the expenditure that made it 
eligible to receive the grant. However, other stake- 
holders said PSAB standards should allow for such 
transfers to be brought into revenue over time as the 
tangible capital asset acquired or constructed with 
the transferred funds is amortized. 

After substantial discussion, consideration 
of respondents' views, and the issue of several 
documents for comments, PSAB approved a new 
Government Transfers standard in December 2010. 
Under the new standard, a recipient government 
must recognize a transfer as revenue when the 
transfer has been authorized and has met all eligi- 
bility criteria, with one exception. This requirement 
does not apply when the transferring government 
creates a liability for the recipient government by 
imposing stipulations on the use of the transfer or 
on the actions the recipient needs to take to keep 
the transfer. As well, the standard recognizes that 
actions and communications by the recipient that 
restrict the use of the funds for a specific future 
purpose can create a liability. In practice, whether 
the facts and circumstances surrounding a particu- 
lar transfer support the recognition of a liability is 
a matter of professional judgment. If a transfer is 
deemed to create a liability for the recipient govern- 
ment, the transfer is recognized as revenue offset- 
ting the expenditure of the funds as the liability is 
settled over time. 

FINANCIAL INSTRUMENTS 

Financial instruments — including derivatives such 
as foreign-exchange forward contracts, swaps, 
futures, and options — are used to manage financial 
risks. Currently, PSAB guidance on accounting for 
the use of derivatives is limited to their applica- 
tion in hedging foreign-currency risks, such as 
government debt denominated in foreign funds. 
Accordingly, Ontario and all other governments in 
Canada provide details on their financial risks, the 
use of financial instruments such as derivatives to 
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mitigate these risks, and the current fair value of 
their reported debt in the notes to their financial 
statements. 

In January 2005, the Accounting Standards 
Board (AcSB) of the Canadian Institute of Char- 
tered Accountants (CICA) approved three new 
accounting standards titled "Financial Instru- 
ments," "Comprehensive Income," and "Hedges." 
These private-sector standards underscored the 
need to address these same issues from a public- 
sector perspective. Accordingly, PSAB created a task 
force to consider how governments should account 
for their financial instruments. One of the key 
issues the task force needed to address was whether 
changes in the market or fair value of derivative 
contracts should be reflected in a government's 
financial statements and, in particular, whether 
they should affect the measurement of the govern- 
ment's annual surplus or deficit. 

A few of the milestones and decision-points over 
the life of the project are highlighted below: 

• The PSAB task force issued a statement of 
principles on financial instruments in June 
2007 that set out suggested principles for the 
recognition and measurement of financial 
instruments consistent with the direction 
provided by the AcSB. 

• Governments, in response to PSAB's proposed 
standard, raised concerns that reflecting 
fair-value changes that do not result in any 
money actually coming in or going out was 
not reflective of the inflows and outflows 

of economic resources associated with the 
delivery of services to the public. A key point 
raised by this group was that most govern- 
ments enter into derivative contracts to hedge 
their foreign-currency or interest-rate risks, 
and therefore hold these contracts until they 
mature, at which time any gains or losses aris- 
ing during the period the derivative was held 
would net to zero. 

• PSAB responded to stakeholder concerns in 
September 2009 with a revised proposal for a 
Financial Instruments standard recommending 



that all unrealized gains and losses from fair- 
value remeasurement of financial instruments 
be recorded in the statement of operations, but 
that these gains and losses should be reported 
separately from other government revenues 
and expenses. PSAB hoped that separate 
reporting of these remeasurement gains and 
losses would clearly distinguish their impact 
on any annual surplus or deficit and thus 
alleviate stakeholder concerns. 
• Responses from all governments to this 
proposal continued to raise concern that the 
inclusion of unrealized market- value gains 
and losses in government financial statements 
would create volatility and not reflect the 
economic substance of government financing 
transactions. As well, we and others disagreed 
with the "two bottom lines" that the proposed 
standard would require including in a govern- 
ment's consolidated statement of operations, 
and felt this would be confusing to users of 
the statements. 
PSAB responded by proposing a new stan- 
dard in November 2010. Its main requirements 
included recording derivatives at fair value and 
recording unrealized changes in their fair value 
in a new Statement of Remeasurement Gains and 
Losses. Unrealized gains and losses from fair-value 
remeasurement of financial instruments would not 
be recorded in the statement of operations. Consist- 
ent with the previous proposal, hedge accounting 
would no longer be required. 

In response to this proposal, we indicated that 
our primary focus in assessing any proposed change 
to accounting standards is to consider its impact on 
the determination of a government's annual surplus 
or deficit. Therefore, we said we firmly believed 
that measurable changes in the value of assets and 
liabilities that occur under a government's watch 
should generally be included in their reported sur- 
plus or deficit in the period these changes occurred. 
However, we did not support reporting the annual 
changes in the values of financial instruments in a 
separate quasi-equity statement. In our view, the 
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addition of a separate statement of remeasurement 
gains and losses would diminish the value of a 
government's statement of operations. Also, adding 
yet another statement would do little to make a 
government's financial statements, which already 
tend to be quite complex, more understandable to 
the lay reader. 

We proposed that any new financial instruments 
standard should recognize the fundamental dif- 
ference between derivatives acquired to mitigate 
foreign-currency risk and those acquired to mitigate 
interest-rate risk. Specifically, we proposed that 
PSAB consider recording only foreign-currency 
derivatives at fair value because changes in the fair 
value of both the debt and the offsetting derivative 
would be recorded in the government's statement 
of operations and thereby affect the annual surplus 
or deficit appropriately. However, because changes 
in the fair value of interest-rate derivatives would 
result in only a one-sided valuation change being 
recorded in the statement of operations, we pro- 
posed including changes in their fair value only in 
the notes to the financial statements and not in the 
statement of operations. 

However, after substantial discussion and con- 
sideration of respondents' views, PSAB approved 
the new Financial Instruments standard in March 
2011, reflecting the proposals made in its November 
2010 re-exposure draft. We appreciated the oppor- 
tunity to present our views on this issue and, even 
though PSAB chose not to accept the alternative 
we had proposed, we accept the final standard and 
will continue to apply all of PSAB's standards in 
auditing the fairness of the province's consolidated 
financial statements. 



Public Sector Accounting 
Board Initiatives 



This section briefly outlines some of the other more 
significant issues that the Public Sector Accounting 



Board (PSAB) of the Canadian Institute of Char- 
tered Accountants (CICA) has been dealing with 
over the last year, which may in future affect the 
province's consolidated financial statements. 

INTRODUCTION 

As noted earlier, PSAB has the authority to set 
accounting and financial reporting standards for 
the public sector in Canada. In addition to issu- 
ing revised standards for financial instruments 
and government transfers discussed earlier in 
this chapter, some of the other more noteworthy 
financial accounting and reporting issues PSAB 
resolved during the past year include determin- 
ing the appropriate accounting framework for 
government organizations and the accounting 
for foreign-currency translation. One of the more 
significant projects that PSAB is currently working 
on is a revision of its conceptual framework that 
supports the development of consistent accounting 
standards for the public sector in Canada. 

STANDARDS 

Government Not-for-profit Organizations 

The CICA's Accounting Standards Board (AcSB) is 
responsible for establishing Canadian accounting 
and financial-reporting standards for private-sector 
profit-oriented enterprises and private-sector 
not-for-profit organizations. In response to the 
ongoing globalization of financial markets and the 
movement toward worldwide standards, the AcSB 
implemented a number of financial reporting chan- 
ges this year. 

International Financial Reporting Standards 
(IFRS) replaced the previous set of Canadian gener- 
ally accepted accounting principles (GAAP) as the 
accounting framework used to prepare the financial 
statements of publicly accountable, profit-oriented 
enterprises. For enterprises that are not publicly 
accountable or profit-oriented and which did not 
wish to adopt IFRSs, the AcSB introduced two 
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additional accounting frameworks: "Accounting 
Standards for Private Enterprises" and "Account- 
ing Standards for Private Sector Not-for-Profit 
Organizations." 

In September 2010, PSAB concluded that 
government not-for-profit organizations should 
apply the provisions of the Public Sector Accounting 
Handbook (PSA Handbook) . To ease the transition 
of these organizations into the PSA Handbook, 
PSAB introduced specific not-for-profit standards, 
known as the "4200 Series," in the PSA Handbook. 
These new standards are for the most part similar 
to Canadian GAAP used previously by not-for-profit 
organizations. A not-for-profit organization can 
elect to follow the 4200 Series in the PSA Handbook 
or, alternatively, apply the provisions of the PSA 
Handbook without the 4200 Series. This require- 
ment is effective for fiscal periods beginning on or 
after January 1, 2012. 

Foreign-currency Translation 

In June 2011, PSAB also issued a new accounting 
standard on foreign-currency translation to ensure 
consistency with its new standard on financial 
instruments. Although the revised standard 
addresses a number of issues, the most significant 
revision eliminates the current requirement to 
defer and amortize gains and losses resulting 
from foreign-exchange fluctuations. Similar to the 
new standard on financial instruments, unreal- 
ized foreign-exchange gains and losses will now 
be recorded separately from other revenues and 
expenses of a government or its organization, in 
the Statement of Remeasurement Gains and Losses. 
Only when the actual gains and losses from foreign 
exchange fluctuations are realized will they be 
recorded in the Statement of Operations and hence 
impact the surplus or deficit of the government or 
its organization. This revised standard is also effect- 
ive for fiscal periods beginning on or after April 1, 
2012, for government organizations and April 1, 
2015, for governments. 



Conceptual Framework 

As indicated earlier, PSAB's conceptual framework 
is a set of interrelated objectives and fundamen- 
tals that support the development of consistent 
accounting standards. It is the basis on which stake- 
holders such as those who prepare government 
financial statements, legislative auditors, and PSAB 
discuss and assess proposals to address emerging 
accounting issues. A key benefit of the framework is 
to instill discipline into the standard-setting process 
to ensure that accounting standards are objective, 
credible, and consistent. 

PSAB formed the Conceptual Framework Task 
Force in response to concerns raised by several gov- 
ernments regarding current revenue and expense 
definitions, which they contend result in volatility 
in reported results and make budget-to-actual com- 
parisons difficult. The objective of the Task Force 
is to review the appropriateness of the concepts 
and principles in the existing Framework for the 
public sector in the PSA Handbook. In April 2011, 
the Task Force began its review of the conceptual 
framework. The following August, it issued the first 
of two consultation papers to seek input from stake- 
holders on the key characteristics of public-sector 
entities and their accounting and reporting implica- 
tions for general-purpose financial statements. The 
Task Force plans to issue a second consultation 
paper in the second quarter of 2012 that will likely 
discuss the following issues related to financial 
statements: 

• users and what they need to have reported; 

• the objectives of reporting in financial state- 
ments; and 

• the qualitative characteristics of information 
to be reported. 

The input received from the two consultation 
papers will then be considered in drafting a state- 
ment of principles for public comment. 
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Statutory Matters 



Under section 12 of the Auditor General Act, I am 
required to report on any Special Warrants and 
Treasury Board Orders issued during the year. In 
addition, section 91 of the Legislative Assembly Act 
requires that I report on any transfers of money 
between items within the same vote in the Esti- 
mates of the Office of the Assembly 

LEGISLATIVE APPROVAL OF 
EXPENDITURES 

Shortly after presenting its budget, the govern- 
ment tables detailed Expenditure Estimates in the 
Legislative Assembly outlining, on a program-by- 
program basis, each ministry's spending proposals. 
The Standing Committee on Estimates (Committee) 
reviews selected ministry estimates and presents 
a report on them to the Legislature. The estimates 
of those ministries that are not selected for review 
are deemed to be passed by the Committee and are 
so reported to the Legislature. Orders for Concur- 
rence for each of the estimates reported on by the 
Committee are debated in the Legislature for a 
maximum of two hours and then voted on. 

Once the Orders for Concurrence are approved, 
the Legislature provides the government with legal 
spending authority by approving a Supply Act, 
which stipulates the amounts that can be spent by 
ministry programs, typically those set out in the 
estimates. Once the Supply Act is approved, the 
individual program expenditures are considered to 
be Voted Appropriations. The Supply Act pertaining 
to the fiscal year ended March 31, 2011, received 
Royal Assent on March 30, 2011. 

The Supply Act is typically not passed until 
well after the start of the fiscal year, but ministry 
programs require interim spending authority prior 
to its passage. For the fiscal year ending March 31, 
2011, the Legislature authorized these payments 
by passing three acts allowing interim appropria- 



tions: the Interim Appropriation for 2010-2011 Act, 
2009; the Supplementary Interim Appropriation Act, 
2010; and the Supplementary Interim Appropriation 
Act, 2010 (No. 2). These three acts received Royal 
Assent on December 15, 2009, May 18, 2010, and 
December 8, 2010, respectively, and authorized the 
government to incur up to $123.8 billion in public- 
service expenditures, $3.7 billion in investments 
of the public service, and $176 million in legisla- 
tive office expenditures. All three acts were made 
effective as of April 1, 2010. On February 23, 2011, 
the Legislature also passed a motion of interim 
supply providing the legislative offices with tempor- 
ary approval to incur the additional expenditures 
contained in the 2010/11 Estimates that were not 
authorized under the three interim acts. 

The three interim acts, in conjunction with the 
motion of interim supply, provided the government 
with sufficient temporary appropriations to allow it 
to incur expenditures from April 1, 2010, to when 
the Supply Act received Royal Assent on March 30, 
2011. As the legal spending authority under the 
interim acts was intended to be temporary, all three 
were repealed under the Supply Act, 2011, and the 
authority to incur expenditures provided under 
them was subsumed into the authority provided 
under the Supply Act, 2011 . 

SPECIAL WARRANTS 

If the Legislature is not in session, section 1.0.7 of 
the Financial Administration Act allows for the issu- 
ance of Special Warrants authorizing the incurring 
of expenditures for which there is no appropriation 
by the Legislature or for which the appropriation 
is insufficient. Special Warrants are authorized by 
Orders-in-Council approved by the Lieutenant Gov- 
ernor on the recommendation of the government. 

There were no Special Warrants issued for the 
fiscal year ended March 31, 2011. 



Public Accounts of the Province 



TREASURY BOARD ORDERS 

Section 1.0.8 oi the Financial Administration Act 
allows the Treasury Board to make an order author- 
izing expenditures to supplement the amount of 
any voted appropriation that is expected to be 
insufficient to carry out the purpose for which 
it was made. The order may be made only if the 
amount of the increase is offset by a corresponding 
reduction of expenditures to be incurred from other 
voted appropriations not fully spent in the fiscal 
year. The order may be made at any time before 
the books of the government for the fiscal year are 
closed. The government considers the books to be 
closed when any final adjustments arising from our 
audit have been made and the Public Accounts have 
been tabled in the Legislature. 

Even though the Treasury Board Act, 1991 
was repealed and re-enacted within the Financial 
Administration Act in December 2009, subsection 
5 (4) of the repealed act allows the Treasury Board 
to delegate to any member of the Executive Council 
or to any public servant employed under the Public 
Service of Ontario Act, 2006 any power, duty, or 
function of the Treasury Board, subject to limita- 
tions and requirements that the Treasury Board 
may specify. This delegation under the repealed 
act will continue to be in effect until replaced by a 
new delegation. For the fiscal year ended March 31, 
2011, the Treasury Board delegated its authority 
to ministers for issuing Treasury Board Orders to 
make transfers between programs within their min- 
istries, and to the Chair of the Treasury Board for 
making transfers in programs between ministries 
and making supplementary appropriations from 
contingency funds. Supplementary appropriations 
are Treasury Board Orders in which the amount 
of an appropriation is offset by a reduction to the 
amount available under the government's centrally 
controlled contingency fund. 

Figure 8 summarizes the total value of Treasury 
Board Orders issued for the past five fiscal years. 

Treasury Board Orders increased significantly in 
the 2009/10 fiscal year, primarily in the Ministry of 



Figure 8: Total Value of Treasury Board Orders, 
2006/07-2010/11 ($ million) 

Source of data: Treasury Board 
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Finance and the Ministry of Energy and Infrastruc- 
ture (now the two separate ministries of Energy and 
of Infrastructure), as a result of loans to the auto 
sector and infrastructure stimulus spending. Figure 
9 summarizes Treasury Board Orders for the fiscal 
year ended March 31, 2011, by month of issue. 

According to the Standing Orders of the Legis- 
lative Assembly, Treasury Board Orders are to 
be printed in the Ontario Gazette, together with 
explanatory information. Orders issued for the 
2010/11 fiscal year are expected to be published in 
the Ontario Gazette in December 2011. A detailed 
listing of 2010/11 Treasury Board Orders, showing 
the amounts authorized and expended, is included 
as Exhibit 3 of this report. 

TRANSFERS AUTHORIZED BY THE 
BOARD OF INTERNAL ECONOMY 

When the Board of Internal Economy authorizes 
the transfer of money from one item of the Esti- 
mates of the Office of the Assembly to another item 
within the same vote, section 91 of the Legislative 
Assembly Act requires that we make special mention 
of the transfer(s) in our Annual Report. 

With respect to the 2010/11 Estimates, there 
were no transfers made within the votes of the 
Office of the Assembly. 
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Figure 9: Total Value of Treasury Board Orders by 
Month in 2010/11 

Source of data: Treasury Board 
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UNCOLLECTIBLE ACCOUNTS 

Under section 5 of the Financial Administration 
Act, the Lieutenant Governor in Council, on the 
recommendation of the Minister of Finance, may 
authorize an Order-in-Council to delete from the 
accounts any amounts due to the Crown that are 
the subject of a settlement or deemed uncollectible. 
The amounts deleted from the accounts during any 
fiscal year are to be reported in the Public Accounts. 

In the 2010/11 fiscal year, receivables of 
$432.1 million due to the Crown from individuals 
and non-government organizations were writ- 
ten off (the comparable amount in 2009/10 was 
$410.3 million). The major portion of the writeoffs 
in the 2010/11 fiscal year related to the following: 



• $145.2 million for uncollectible receiv- 
ables under the Student Support Program 
($316.7 million in 2009/10); 

• $118.8 million for uncollectible receivables 
under the Ontario Disability Support Program 
($5 million in 2009/10); 

• $71.9 million for uncollectible retail sales tax 
($21.4 million in 2009/10); 

• $65.1 million for uncollectible corporate tax 
($55.5 million in 2009/10); 

• $9.6 million for uncollectible tobacco tax 
($200,000 in 2009/10); and 

• $6.4 million for uncollectible employer health 
tax ($5.4 million in 2009/10). 

Volume 2 of the 2010/11 Public Accounts 
summarizes the writeoffs by ministry. Under the 
accounting policies followed in the preparation of 
the consolidated financial statements of the prov- 
ince, a provision for doubtful accounts is recorded 
against accounts receivable balances. Accordingly, 
most of the writeoffs had already been expensed 
in the government's consolidated financial state- 
ments. However, the actual deletion from the 
accounts required Order-in-Council approval. 
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Reports on 

Value-for-money Audits 
and Reviews 



Our value-for-money (VFM) audits are intended to 
examine how well government, organizations in 
the broader public sector, agencies of the Crown, 
and Crown-controlled corporations manage their 
programs and activities. These audits are conducted 
under subsection 12(2) of the Auditor General Act, 
which requires that the Office report on any cases 
observed where money was spent without due 
regard for economy and efficiency or where appro- 
priate procedures were not in place to measure 
and report on the effectiveness of service delivery. 
Where relevant, such audits also encompass com- 
pliance issues. This chapter contains the conclu- 
sions, observations, and recommendations for the 
value-for-money audits and one review conducted 
in the past audit year. 

The ministry programs and activities and the 
organizations in the broader public sector audited 
this year were selected by the Office's senior man- 
agement on the basis of various criteria, such as 
a program's or organization's financial impact, its 
perceived significance to the Legislative Assembly, 
related issues of public sensitivity and safety, and 
the results of past audits and related follow-up 
work. 

We plan, perform, and report on our value-for- 
money work in accordance with the professional 



standards for assurance engagements, encompass- 
ing value for money and compliance, established by 
the Canadian Institute of Chartered Accountants. 
Accordingly, our audits include such tests and other 
procedures as we consider necessary in the circum- 
stances, including obtaining advice from external 
experts when needed. 

Before beginning an audit, our staff conduct 
in-depth research into the area to be audited and 
meet with auditee representatives to discuss the 
focus of the audit. During the audit, staff maintain 
an ongoing dialogue with the auditee to review the 
progress of the audit and ensure open lines of com- 
munication. At the conclusion of the audit field- 
work, which is normally completed by late spring of 
that audit year, a draft report is prepared, reviewed 
internally, and then discussed with the auditee. 
Senior Office staff meet with senior management 
from the auditee to discuss the draft report and to 
finalize the management responses to our recom- 
mendations. In the case of organizations in the 
broader public sector, discussions are also held with 
senior management of the funding ministry. All 
responses are then incorporated into the report in 
each of the VFM sections. 
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Financial Services Commission of Ontario 

Section 

3.01 Auto Insurance 



Regulatory Oversight 



Background 



The Financial Services Commission of Ontario 
(FSCO) is an arm's-length agency of the Ministry 
of Finance responsible for regulating the province's 
insurance sector, including auto insurance. FSCO 
also regulates pension plans, mortgage brokers, 
credit unions, caisses populaires, loan and trust 
companies, and co-operative corporations in 
Ontario. 

FSCO's mandate is to provide regulatory services 
that protect the public interest and enhance public 
confidence in the regulated sectors through licens- 
ing, monitoring, and enforcement. FSCO's senior 
official, the Superintendent of Financial Services, 
is responsible for the general supervision of the 
regulated sectors as well as the administration and 
enforcement of the Financial Services Commission of 
Ontario Act and other related statutes. 

The most significant piece of legislation for auto 
insurance is the Insurance Act, which establishes 
standards for the auto insurance industry and 
empowers FSCO to regulate insurer behaviour and 
investigate complaints about unfair practices. 

FSCO's high-profile activities include ruling on 
applications for premium-rate changes by Ontario's 
100 or so private-sector insurance companies. 
About 20 of these companies hold about 75% of the 



market. Commission rulings must ensure that the 
proposed premiums are justified based on factors 
such as an insurance company's past and expected 
claim costs, its operating expenses, and what would 
be a reasonable profit. 

In addition, FSCO periodically reviews the statu- 
tory accident benefits available to people injured in 
automobile accidents. It provides dispute resolution 
services, such as mediation, to settle disagreements 
between insurers and injured persons about the 
entitlement to and amount of statutory accident 
benefits. FSCO also administers the Motor Vehicle 
Accident Claims Fund, which compensates people 
injured in automobile accidents when there is 
no insurer to cover the claim. The Fund is mainly 
financed by revenues from fees for drivers' licence 
registrations and renewals. 

In the 2010/11 fiscal year, FSCO spent a total of 
$59 million. Expenditures for FSCO's Auto Insur- 
ance Division were approximately $14 million, with 
95% of that amount going to salaries and benefits. 
FSCO recovers all of its costs relating to the regula- 
tion of auto insurance from insurance companies 
operating in Ontario. 

Auto Insurance in Ontario 

Ontario has about 9 million licensed drivers and 
7.5 million passenger cars and trucks. In the past 
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10 years, the number of people killed or injured 
in motor vehicle accidents in the province has 
declined by about 25%. In 2009, the latest year for 
which a breakdown exists, 535 people were killed 
in accidents and another 61,975 were injured. 
Approximately 60% of injuries were minor, includ- 
ing sprains, strains, and minor or moderate whip- 
lash, while 39% were moderate to major, including 
fractures or internal organ damage. The remaining 
1% — about 800 people — suffered catastrophic 
injuries, such as severe brain impairment or para- 
plegia, or required amputation. 

Auto insurance has been compulsory in Ontario 
since 1979. In 1990, the province introduced a 
mixed no-fault/tort insurance system, requiring 
the payment of injury and property-damage claims 
by the insurance company of each vehicle involved 
in an accident, regardless of fault. Coverage levels 
for different types of injuries and death claims are 
set out in the Statutory Accident Benefits Schedule 
(SABS) under the Insurance Act. However, people 
experiencing serious injuries can also sue at-fault 
drivers for damage in excess of SABS benefits for 
economic loss and/or pain and suffering. 

Despite the no-fault rules, Ontario law requires 
insurers to assign "fault" to a driver in an accident 
as set out in regulations to the Act, which can lead 
to increases in that driver's premiums. 

Ontario motorists are required to purchase 
insurance that includes: 

• standard SABS coverage for medical benefits, 
attendant care, and disability income for 
people injured in an automobile accident as 
well as death and funeral benefits for those 
killed in an accident regardless of who was at 
fault; 

• a minimum of $200,000 in third-party liability 
coverage for personal and property claims as a 
result of lawsuits against the insured; 

• direct compensation coverage for damage 
to a vehicle owned by the insured caused by 
another driver (no fault) ; and 



• uninsured automobile coverage to protect 
against injuries and damage to a vehicle 
caused by an uninsured motorist. 

Consumers can increase their third -party liabil- 
ity and SABS coverage and also purchase other 
optional insurance, such as caregiver coverage. 
Additional voluntary insurance coverage for the 
vehicle is also available, including collision cover- 
age for damage to vehicles and comprehensive 
coverage for theft, vandalism, and other perils such 
as fire, flood, or hail. For example, FSCO informed 
us that 99% of Ontario drivers in the five years end- 
ing in 2010 purchased more than the mandatory 
$200,000 minimum third-party liability coverage. 

In the 2010 calendar year, Ontario drivers paid 
$9.8 billion in auto insurance premiums. The 
total number of claims in 2010 was approximately 
584,000, with claims costs totalling $8.7 billion, 
broken down as follows: 

• $4.5 billion in SABS benefits; 

• $2 billion for third-party liability; 

• $900 million in direct compensation for prop- 
erty damage caused by other drivers; and 

• $1.3 billion for other property claims such as 
collision and comprehensive damage. 



Audit Objective and Scope 



Our audit objective was to assess whether FSCO 
had adequate systems and procedures in place with 
respect to its auto insurance responsibilities to: 

• ensure compliance with relevant legislation 
and its own policies established to protect the 
public interest and to enhance public confi- 
dence in the auto insurance sector; 

• administer the Motor Vehicle Accident Claims 
Fund in the public interest; and 

• measure and report on the effectiveness of its 
regulatory oversight. 

Prior to our fieldwork, we identified criteria 
to be used to address our audit objective. Senior 
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management at FSCO reviewed these criteria and 
agreed to them. 

The scope of our audit included a review and 
analysis of FSCO's relevant files, policies, and pro- 
cedures, as well as interviews with the appropriate 
staff. We also held discussions with, and obtained 
information from, a variety of organizations, 
including insurance companies, the Insurance Bur- 
eau of Canada (the national industry association 
representing some 90% of the private insurance 
market), and other stakeholders such as health-care 
providers, consumers, and lawyers with an interest 
in auto insurance. 

We researched auto insurance regulatory legis- 
lation and operations in several other North Amer- 
ican jurisdictions and visited the Manitoba Public 
Insurance Corporation, the Insurance Corporation 
of British Columbia, the Alberta Superintendent of 
Insurance, and the Alberta Automobile Insurance 
Rate Board to discuss their perspectives on regulat- 
ing the auto insurance sector and the administra- 
tion of insurance operations. We also engaged on 
an advisory basis the services of an independent 
expert with senior management experience in the 
insurance sector. 

We also reviewed recent audit reports issued by 
the government's Finance and Revenue Audit Servi- 
ces Team related to FSCO and, as a result, we were 
able to reduce the scope of our examination over 
the Motor Vehicle Accident Claims Fund's contract 
with an independent claims adjuster. 




The responsibility of the government includes bal- 
ancing the need for a financially stable auto insur- 
ance sector with the need to ensure that consumers 
pay affordable and reasonable premiums and 
receive fair and timely benefits and compensation 
when they are involved in accidents. The Super- 
intendent of Financial Services (Superintendent) 
is responsible for administering the legislation 



and regulations that the government establishes 
to achieve these objectives. Claims payments are 
the largest driver of the cost of auto insurance 
premiums, and with the average injury claim in 
Ontario of about $56,000 being five times more 
than the average claim in other provinces, Ontario 
drivers generally pay much higher premiums than 
other Canadian drivers do. Another reason claims 
costs in Ontario are higher is because Ontario's 
coverage provides for one of the most comprehen- 
sive and highest benefit levels in Canada. 

Although the government has begun taking 
action to address the high cost of claims in Ontario, 
the following observations outline some of the 
challenges the Financial Services Commission of 
Ontario (FSCO) faces if it is to be more successful in 
proactively fulfilling its role of protecting the public 
interest and enhancing public confidence in the 
auto insurance industry. 

• From 2005 to 2010, the total cost of injury 
claims under the Statutory Accident Benefits 
Schedule (SABS) rose 150% even though 
the number of injury claims in the same 
period increased only about 30%. Moreover, 
the number of injury claims in 2009, at 
almost 75,000, was 20% higher than the 
number of people reported by the Ministry 
of Transportation as having been injured in 
automobile accidents that year and FSCO had 
not analyzed the reasons for this significant 
difference. 

• Between 2008 and 2009, SABS benefits pay- 
ments rose 37% in the Greater Toronto Area 
(GTA), compared to 23% in other Ontario 
cities and just 14% in rural areas. According 
to FSCO this may be attributable in part to 
the concentration of plaintiff representor and 
health-care provider communities in the GTA. 
Accordingly, GTA vehicle owners pay higher 
premiums than motorists in other parts of 
Ontario. 

• FSCO had not routinely obtained assur- 
ances from insurance companies — nor had 
it conducted any regular on-site compliance 
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reviews to ensure — that they have paid the 
proper amounts for claims or that they have 
handled claims judiciously. Without such 
assurances, the risk exists that consumers will 
not be treated fairly There is also a risk that 
unnecessarily high claims costs could result in 
the need for insurers to raise premiums and 
may also help insurers obtain approval from 
FSCO for higher premium increases. FSCO has 
recently initiated action to address this. 
Industry estimates peg the value of auto 
insurance fraud in Ontario at between 10% 
and 15% of the value of 2010 premiums, or as 
much as $1.3 billion. Unlike many other prov- 
inces and American states, Ontario does not 
have significant measures in place to combat 
fraud. The government and FSCO are await- 
ing the recommendations of a government- 
appointed anti-fraud task force expected in 
fall 2012. 

In approving premium rates for individual 
insurance companies, FSCO allows insurers a 
reasonable rate of return, which was origin- 
ally set at 12.5% in 1988, based on the bench- 
mark long-term bond rate of 10%, and revised 
to 12% in 1996. However, that profit margin 
has not been adjusted downward since that 
time, even though the long-term bond rate 
has been about 3% for the last couple of years 
and is projected to remain at a relatively low 
level for some time. Furthermore, FSCO needs 
to improve its documentation supporting its 
premium-rate-change decisions and approvals 
to ensure that it can demonstrate that it treats 
all insurers' requests consistently and that 
premium-rate changes approved are just and 
reasonable. 

Increasing demand and restraints on resour- 
ces have caused significant backlogs in FSCO's 
mediation services for claimants in dispute 
with insurers, with resolutions taking 10 to 12 
months rather than the legislated 60 days. It 
also did not capture information that would 
allow it to assess the reasons why the number 



of applications for mediation has sharply 
risen — by 135% over the last five years, with 
about half of all injury claims ending up in 
mediation. Demand for mediation is high- 
est in the GTA, where 80% of all mediation 
applications originate, even though the GTA 
accounts for just 45% of automobile accidents 
involving injuries. 

• FSCO does not yet have any meaningful 
measures of its success in meeting its mandate 
to oversee auto insurance or of its customer 
service performance that could be publicly 
reported in its annual report and on its 
website. 

We considered FSCO's first comprehensive 
review of the statutory accident benefits, which was 
completed in 2009, to have been a good measure 
to assess automobile injury claims, although we 
believe that such reviews should be conducted 
when circumstances warrant doing so rather than 
only at the legislated five-year frequency. As a result 
of the first review, the SABS was changed by the 
government in September 2010. FSCO advised us 
that it was too early to determine if the changes had 
mitigated the significant recent growth in the aver- 
age claim cost and stabilized premiums. 

Related areas that the government and FSCO 
needed to address include the following: 

• The Motor Vehicle Accident Claims Fund had 
$109 million less in assets as of March 31, 
2011, than it needs to satisfy the estimated 
lifetime costs of all claims currently in the 
system. This unfunded liability is expected 
to triple by the 2021/22 fiscal year unless 
the revenues are significantly increased. For 
instance, the government would have to 
double the $15 fee currently added to every 
driver's licence renewal to eliminate the 
unfunded liability. 

• All provinces, including Ontario, require 
that insurers, rather than taxpayers, pay for 
the health-care-system costs of automobile- 
accident victims. The amount of assessment 
FSCO collects annually from insurers on behalf 
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of the Ministry of Health and Long-Term Care 
to cover these costs has not increased since 
2006, even though health-care spending in 
Ontario has increased by about 25% and 
medically-related statutory accident benefit 
costs have increased by almost 120% over the 
same period. If Ontario's health-care assess- 
ment per registered vehicle were raised to the 
average of other provinces, the cost to the tax- 
payers of covering these health-care expenses 
would be reduced by more than $70 million, 
but such a move would likely add almost $10 
to the annual insurance premium for each 
vehicle in Ontario. 

| OVERALL FSC0 RESPONSE 

FSCO welcomes the Auditor General's recom- 
mendations. While the effectiveness and admin- 
istration of Ontario's auto insurance regulatory 
regime by FSCO is generally sound, the audit 
recommendations will strengthen the oversight 
of the auto insurance system. 

The government has a challenging task in 
balancing the need for a financially stable auto 
insurance sector with the needs of consumers. 
FSCO supports the government in meeting this 
challenge by administering auto insurance legis- 
lation and regulations. FSCO plays an important 
role in ensuring that the pricing of auto insur- 
ance in Ontario remains reasonable through 
its rate regulation process and that individuals 
injured in auto accidents are treated fairly. 

In 2009, FSCO completed its first com- 
prehensive five-year review of Ontario's auto 
insurance system, which it presented to the gov- 
ernment. The review assessed several systemic 
problems and, as a result of the first review, the 
government made significant regulatory chan- 
ges in September 2010. FSCO continues to work 
on implementing a range of additional longer 
term projects announced by the government as 
part of its 2010 reforms. 



STATUTORY ACCIDENT BENEFITS 
CLAIMS COSTS 

Past Reforms 

Ontario's auto insurance program has undergone 
numerous changes since the introduction of a mixed 
no-fault/tort insurance system in 1990, with legisla- 
tive reforms enacted in 1994, 1996, 2003, 2006, and 
2010. These changes were made largely to address 
both the significant growth in the cost of Statutory 
Accident Benefits Schedule (SABS) payouts and the 
resulting increase in insurance premiums. In each 
case, however, the legislative reforms provided only 
temporary relief from higher premiums. As a result, 
we noted that Ontario's auto insurance system has a 
history of increasing claims costs, which insurance 
providers ultimately pass on to drivers through 
higher premiums. In our view, more timely changes 
could have been made and are still needed to con- 
trol claims costs and premiums. 

In 2003, the government amended the Insurance 
Act to require the Superintendent of Financial Ser- 
vices (Superintendent) to undertake a review of the 
effectiveness and administration of auto insurance 
at least every five years and make recommenda- 
tions for improvement to the Minister of Finance. 
In 2008, FSCO undertook the first statutory five- 
year review, which led to a report to the Minister 
of Finance and to legislative changes in the SABS 
in September 2010. By that time, however, claims 
costs had already risen rapidly between 2005 and 
2010, as shown in Figure 1. From 2005 to 2010, 
total claims costs in Ontario increased by 61%, from 
$5.4 billion to $8.7 billion. FSCO informed us that 
the primary cause for this escalating trend was 
increased SABS benefits costs, not the increase in 
the number of accident claims. Indeed, the injuries 
claim costs rose 150%, even though the number of 
injury claims increased by only 30% over the same 
period. 



Auto Insurance Regulatory Oversight 



Figure 1: Ontario Average Premium and Claim Cost, 
2005-2010 ($ per insured private passenger vehicle) 

Source of data: General Insurance Statistical Agency* 
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The General Insurance Statistical Agency is a not-for-profit corporation es- 
tablished to compile auto insurance statistics on behalf of Superintendents 
in provinces where there is private-insurer delivery of auto insurance. Sta- 
tistics reported include private passenger vehicles and exclude commercial 
vehicles. 



When we analyzed the $8.7 billion in total 
claims costs for 2010, we found that SABS benefits 
costs accounted for $4.5 billion, or more than half 
the total. That compares with 2005 SABS benefits 
costs of $1.8 billion, or one-third of the total. Com- 
pared to the 150% increase in the SABS portion of 
total claims costs between 2005 and 2010, the other 
claims-costs components, including third-party 
liability and collision, rose by a more modest 16%, 
to $4.2 billion from $3.6 billion. 

Over the same five-year period, the average 
SABS benefits cost per claim rose 92%, to $56,092 
from $29,189. In its statutory five-year review, 
FSCO identified significant cost increases of 
between 103% and 505% in the key benefit com- 
ponents of the SABS, as illustrated in Figure 2. 

FSCO attributed the cost increases of SABS 
benefits to what it called "over-utilization," 
especially before the reforms of September 2010. 
Previously, there were few limits on treatment and 
assessment expenses, and those that existed were 
higher than needed. We were informed that pro- 
viders of legal and health-care services may have 
benefited from the lack of properly defined limits by 
over-treating and over-assessing patients. 



Figure 2: Increases in Ontario's Statutory Accident 
Benefits Costs by Type of Benefit, 2005 and 2009 
($ million) 

Source of data: Financial Services Commission of Ontario 
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For example, the Insurance Bureau of Canada 
reported that as much as 30% to 40% of every 
dollar spent in 2007 to treat automobile-accident 
claimants in Ontario went to examinations and 
assessments by regulated health professionals prior 
to initiating benefits and treatment. According to 
FSCO, this level of assessment activity was inconsis- 
tent with that being incurred by the other provinces. 

FSCO further informed us that a dramatic cost 
increase in SABS benefits in the Greater Toronto 
Area (GTA) was a major contributor to the overall 
increase in accident benefit costs in the province 
between 2008 and 2009. Over that single year, 
SABS benefits costs rose 37% in the GTA, compared 
to 23% in other Ontario cities and just 14% in rural 
areas. Accordingly, GTA drivers on average pay sig- 
nificantly higher premiums than motorists in rural 
Ontario. 

2010 Auto Insurance Reforms 

SABS benefits increase for more severe injuries. As 
a result, the government and FSCO need to ensure 
that the definitions of injuries are clear, so that 
insurance companies and claimants can agree on 
the associated benefits for the level of health care 
and amount of compensation to which claimants 
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are legally entitled. Where uncertainty exists, 
claimants may seek, typically with the assistance 
of legal professionals and health-care providers, to 
categorize their injuries as more severe to maximize 
benefits and compensation. 

Following FSCO's statutory review of the SABS, 
which included public consultations, the govern- 
ment announced in November 2009 a package of 
41 reforms that it said would provide more con- 
sumer choice and premium stability. The reforms 
would achieve these goals by controlling claims 
costs, responding to medical over-assessments and 
over-treatment of minor injuries, and simplifying 
the administration of the SABS, as well as making 
certain enhanced benefits optional at additional 
premiums. The reforms aimed at controlling claims 
costs included: 

• introduction of a broader definition of minor 
injuries, called the interim Minor Injury 
Guideline, to replace the existing minor- 
injury guideline, called the Pre-approved 
Framework; 

• introduction of an overall $2,000 limit on the 
cost of all automobile-accident-injury assess- 
ments and a $3,500 minor-injuries-benefits 
limit on the cost of all treatment services and 
assessments combined; 

• lower standard medical and rehabilitation 
benefits for moderate to major injuries, along 
with lower coverage for attendant care and 
income replacement benefits; and 

• elimination of housekeeping, home mainten- 
ance, and care-giving benefits for all but 
catastrophic claims. 

No significant changes were made for claim- 
ants with catastrophic injuries, who continue to be 
eligible for a lifetime maximum of $1 million for 
medical treatment and rehabilitation, and another 
lifetime maximum of $1 million for attendant care. 

Regulations to implement the new reforms took 
effect on September 1, 2010. At the time of our 
audit, FSCO and insurance industry representatives 
told us it was too soon to say if the reforms had 
been effective in limiting claims costs and stabil- 



izing premiums. Most insurers we spoke with said 
it would take at least two years to determine the 
impact of the reforms. 

However, FSCO said that it expected some of the 
reforms to lead to lower claims costs. For example, 
before 2010, under the Pre-approved Framework, 
only whiplash and whiplash-associated injuries 
were classified as minor injuries. As a result, fewer 
than 20% of injuries fell within this lower-cost 
framework. Under the new interim Minor Injury 
Guideline, minor injuries now include sprain, abra- 
sion, laceration, strain, or minor whiplash. FSCO 
informed us that it expects 50% to 60% of all SABS 
benefits claims to fall under this new definition, 
which caps total payouts for minor injuries at 
$3,500. 

Some insurance companies have publicly voiced 
concerns about claimants seeking benefits to which 
they are not entitled. A common insurer complaint 
was that some health-care providers repeatedly 
sought for their clients approval from the insurer 
for treatment plans exceeding the $3,500 limit for 
defined minor injuries under the interim Minor 
Injury Guideline. FSCO informed us that it was not 
surprised by this development, because it expected 
consumers and their representatives to test both the 
system and the resolve of the insurance companies. 

If an insurance company suspects that a claim- 
ant's condition meets the definition of minor injur- 
ies, it can ask the claimant to undergo examination 
by the insurer's health-care professional. For 
example, one insurer said in an industry publication 
that it tracked approximately 500 claimants injured 
after the September 2010 rule changes and found 
that one-third of those who initially requested 
higher treatment benefits were placed under 
the $3,500 limit because an insurer-requested 
examination determined that their injuries met 
the definition of a minor injury. Another insurer 
reported that medical examinations undertaken at 
its request determined that 80% of claimants who 
initially sought compensation beyond the $3,500 
cap were in fact not entitled to it. 
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FSCO informed us during our audit that it will 
monitor how all stakeholders, including insurers 
and health-care providers, apply the interim Minor 
Injury Guideline, and that their compliance with the 
new guideline is essential to lower the cost of acci- 
dent benefits and ultimately stabilize premiums. 

According to FSCO, consumers need a better 
understanding about treatment and rehabilitation 
options, as well as the risks of over-treatment. 
Although the Ministry of Finance recommended 
that health-care providers and insurance companies 
work together to improve consumer awareness and 
expectations regarding treatment and outcomes as 
part of the reforms to the SABS, no such action had 
been taken at the time of our audit. 

FSCO also indicated that it expects to replace 
the interim Minor Injury Guideline in 2013 or 2014 
with a more comprehensive evidence-based treat- 
ment protocol for such injuries, which will focus on 
more efficient and effective treatment outcomes. 

Ongoing Due Diligence Claims 
Management 

The insurance industry assesses its financial health 
in large part by a measure it calls the "incurred loss 
ratio." The ratio is determined by dividing aver- 
age claims costs per vehicle by average premiums. 
According to FSCO, any ratio with a value higher 
than 80% of total claims expenses compared to 
total premium revenues may well result in a loss for 
an insurance company when other administrative 
and overhead costs (minus investment income) 
are factored in — a situation that probably cannot 
continue for an extended period. Ontario's incurred 
loss ratio has recently worsened, rising to 93% in 
2010 from 66% in 2005. 

In addition, according to FSCO's records, the 
incurred loss ratios among the top 40 Ontario auto- 
mobile insurers ranged from 65% to 176% in 2010. 
This may indicate that some insurers are better able 
to manage and limit their claims costs and insurance 
risks than others. Indeed, several stakeholders we 
interviewed said that insurance companies did not 



always apply standard due diligence in adjusting or 
questioning benefit claims under the SABS. 

FSCO informed us that it had expected the 
insurance companies to respond to its September 
2010 reforms by more proactively challenging ques- 
tionable claims. However, FSCO advised us that it 
soon identified actions by certain insurers as well as 
health-care providers that were inconsistent with 
the intent of the reforms. As a result, FSCO issued a 
bulletin in March 2011 reminding insurers of their 
responsibility to challenge questionable or inappro- 
priate claims. According to the bulletin, FSCO was 
"aware that a small group of service providers and 
representatives were continuing to abuse the sys- 
tem." The bulletin goes on to say that "insurers are 
expected to have and use policies and procedures 
that comply with best practices and legislative 
requirements when adjusting all claims." 

Following a government announcement in the 
March 2011 Budget, FSCO made it a strategic prior- 
ity in June 2011 to assess how well insurance com- 
panies implemented the September 2010 reforms to 
ensure that consumers are being treated fairly and 
in accordance with the Act. FSCO intends in future 
to conduct compliance audits of insurance compan- 
ies that appear higher-risk, although no dates have 
been set. FSCO last assessed insurance companies' 
compliance with the SABS benefits using a self- 
assessment questionnaire to all insurance compan- 
ies in 2006. On the basis of the responses, it made 
field visits to some insurers and reported on its 
findings in September 2007. 

Auto Insurance in Other Provinces 

All Canadian provinces have laws requiring man- 
datory auto insurance. Three of them (British 
Columbia, Saskatchewan, and Manitoba) deliver 
insurance through government-owned insurance 
corporations. In Quebec, the government insures 
against injuries and death while private insurers 
cover property damage, liability, and personal 
injury in accidents outside the province. Private- 
sector insurance companies serve the remaining six 
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provinces, including Ontario. Manitoba, Saskatch- 
ewan, and Quebec have no-fault insurance systems. 
Ontario and the six other provinces have a mixed 
no-fault/tort system in which benefits are available 
to injured accident victims from their own insurer 
regardless of fault, and people have the right to sue 
responsible parties for further damages. 

Although Ontario has one of the lowest per- 
capita rates of automobile-accident deaths and 
injuries in the country, it also has the highest aver- 
age premium in Canada, as illustrated in Figure 3, 
which also shows that most other provinces gener- 
ally experienced lower premium increases over the 
last five years — and some actually had premiums 



decrease. Claims costs are another key comparison 
because they constitute the largest cost of any auto 
insurance system. Figure 4 shows that Ontario has 
the highest average total claim cost per insured 
vehicle of any province. 

Although health-care costs and income replace- 
ment and standard accident benefits levels vary 
somewhat across Canada, it could also be argued 
that the average benefit claim cost for automobile- 
accident injuries should be reasonably similar 
regardless of whether the comparison is made 
between the GTA and other cities or between 
Ontario and other provinces. However, although 
in 2005 accident benefits cost the same (about 



Figure 3: Provincial Comparison of Average Premiums, 2006-2010 1 ($ per insured private passenger vehicle) 

Source of data: General Insurance Statistical Agency and provincial insurance corporations 
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$30,000 per claim on average) in the GTA and the 
rest of the province, by 2009 the GTA cost per claim 
had risen to $60,500 — about one-third higher than 
the $45,900 cost per claim for the rest of the prov- 
ince. In addition, as Figure 5 indicates, on average 
Ontario's claims costs under the SABS are signifi- 
cantly higher than the statutory accident benefits 

Figure 4: Provincial Comparison of Average Total Claim 
Costs, 2010*($ per insured private passenger vehicle) 

Source of data: General Insurance Statistical Agency and provincial insurance 
corporations 
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Figure 5: Provincial Comparison of Average Costs per 
Claim for Statutory Accident Benefits, 
2006-2010* ($) 

Source of data: General Insurance Statistical Agency and provincial insurance 
corporations 
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claims costs incurred by other provinces, with most 
provinces paying out less than 25% of Ontario's 
benefits. This is at least partly due to Ontario acci- 
dent benefits and the limits on payouts under the 
SABS, which are generally as high as or higher than 
most other provinces, as illustrated in Figure 6. 



RECOMMENDATION 1 



In order to ensure that the Financial Services 
Commission of Ontario (FSCO) can effectively 
monitor Ontario's auto insurance industry, par- 
ticularly claims costs and premiums, and recom- 
mend timely corrective action to the Minister of 
Finance when warranted, FSCO should: 

• implement regular interim reviews of the 
Statutory Accident Benefits Schedule to 
monitor trends such as unexpected escalat- 
ing claims costs and premiums between the 
legislated five-year reviews, in order to take 
appropriate action earlier, if warranted; 

• monitor ongoing compliance with the 
interim Minor Injury Guideline, expedite the 
work to develop evidence-based treatment 
protocols for minor injuries, and identify and 
address any lack of clarity in its definitions of 
injuries; 

• implement its plans as soon as possible to 
obtain assurance that insurance companies 
are judiciously administering accident claims 
in a fair and timely manner; and 

• examine cost-containment strategies and 
benefit levels in other provinces to determine 
which could be applied in Ontario to control 
this province's relatively high claims costs 
and premiums. 



Ontario's auto insurance system is complex and 
FSCO agrees that the system would benefit from 
more frequent reviews. 

In addition to the five-year review, FSCO 
conducts a legislated review every three years 
of the risk-classification and rate-determination 
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Figure 6: Provincial Comparison of Maximum Statutory Accident Benefits, as of June 2011 * 

Source of data: Financial Services Commission of Ontario and each province's auto insurance provider or regulator 





Ontario Maximum 


Range of Maximum Benefits- 


Range of Maximum Benefits- 




Benefits with 


Other Provinces with 


Other Provinces with Publicly 


Benefit Type 


Private Insurers 


Private Insurers 


Operated Insurance 


medical 


$50,000 for 10 years 


$25,000 for four years to 


$150,000 over lifetime to unlimited 




($1 million over lifetime 


$50,000 for four years 


benefit over lifetime 




for catastrophic injury) 






attendant care 


$36,000 for two years 
($1 million over lifetime 
for catastrophic injury) 


included under medical benefits 


included under medical benefits to 
$4,142 per month with no lifetime limit 


income replacement 


70% of gross income 


$0 to $250/week for up to 104 


between 75% of gross income (to a 


-partial disability 


to a maximum of 
$400/week, maximum 
104 weeks 


weeks 


maximum of $300/week for 104 weeks) 
and 90% of net income (up to $83,000) 


income replacement 


partial disability 


$400/week for up to 104 weeks 


partial disability payments continued for 


-full disability 


payments continued for 
lifetime 


to $250/week continued for 
lifetime 


lifetime 


death benefit 


$25,000 to spouse 


spouse: $10,000-$50,000 


spouse: between $5,000 plus $145/week 




and $10,000 to each 


each dependent: $l,000-$6,000 


for 104 weeks and $415,000 




dependent 




each dependent: between $1,000 plus 
$35/week for 104 weeks and $54,817 



* Includes lowest and highest maximum statutory accident benefits provided by the group. Only Ontario and provinces with publicly operated no-fault 
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regulations. As well, FSCO participates in a 
review of the adequacy of the Statutory Acci- 
dent Benefits every two years. FSCO believes 
these three statutory reviews could be combined 
into one comprehensive review that takes place 
on a more frequent basis than every five years 
and will communicate this to the Ministry of 
Finance. As part of a more comprehensive and 
frequent review, FSCO would also examine cost 
containment strategies and benefit levels in 
other provinces and would provide this analysis 
to the government. 

FSCO believes that the development of an 
evidence-based treatment protocol for minor 
injuries is an important step in ensuring that 
the treatment provided to individuals injured 
in auto accidents in Ontario reflects the current 
medical science. FSCO will be issuing a Request 
for Proposal for consulting services to develop 
a new treatment protocol and will be requiring 



that this work be completed in two years instead 
of the planned three-year time frame. 

FSCO also recognizes the importance of 
making insurance companies more accountable 
for the administration of statutory accident 
benefits claims in a fair and timely manner. Dur- 
ing the summer of 2011, FSCO introduced a new 
requirement that insurance companies provide 
CEO attestations that they have controls, pro- 
cedures, and processes in place to ensure com- 
pliance with legislative requirements around the 
payment of such claims. 

FRAUD IN AUTO INSURANCE 

Ontario's Experience 

It is a federal Criminal Code offence to defraud or 
attempt to defraud an insurance company, with 
conviction carrying large fines and/or a prison term 
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of up to 10 years. Auto insurance fraud can involve 
claimants in a legitimate minor accident who 
misrepresent the injury or damage to get higher 
compensation; service providers who claim for 
unnecessary services or for services not rendered; 
and staged accidents and faked injuries. According 
to the Insurance Bureau of Canada and FSCO, a 
significant amount of fraud involves false claims for 
physical injury and accident benefits authorized by 
health-care service providers. 

It is impossible to give a precise figure for the 
value of auto insurance fraud in Ontario, but recent 
insurance industry estimates indicate that the 
problem is serious and suggest that fraud-related 
costs may have accounted for between 10% and 
15% of auto insurance premiums in 2010, or up 
to $1.3 billion. Stated another way, fraud-related 
costs account for up to 15 cents of every dollar of 
premiums paid. 

Another indicator of possible fraud in the system 
is the recent significant discrepancy in the number 
of injury claims reported by the General Insurance 
Statistical Agency compared to the number of injur- 
ies reported by the Ministry of Transportation. Over 
a one-year period in 2009, the number of injury 
claims increased 13% and the average cost of claims 
rose 32%, although the number of reported injuries 
in Ontario from automobile accidents decreased 
by 1%. Moreover, there were almost 75,000 injury 
claims filed — 20% more than the 62,000 injuries 
from automobile accidents actually reported at the 
time of the accidents. Before 2009, the number 
of injury claims was below or slightly above the 
Ministry of Transportation reported injuries. FSCO 
has not analyzed the reason for these significant 
discrepancies and increases, and whether they may 
be partly attributable to fraud. 

Our discussions with insurance industry repre- 
sentatives in Ontario and other provinces indicated 
that the problem of fraud is worse in Ontario than 
elsewhere in Canada, and it has been growing since 
the mid-1990s. Even a decade ago, the Insurance 
Bureau of Canada reported that Ontario had the 



highest fraud rate of the nine provinces that partici- 
pated in a 2001 study. 

Insurers and their customers are the victims 
through increased premiums when auto insur- 
ance fraud is perpetrated. However, the decision 
to investigate fraud is left to each insurer. Most, if 
not all, insurers as well as the Insurance Bureau of 
Canada have their own investigators. FSCO, on the 
other hand, has had a minimal role in fraud identi- 
fication, investigations, and prosecutions. 

If an insurance company decides to take action 
against someone it suspects of fraud, it may contact 
FSCO directly or pass information on to the Insur- 
ance Bureau of Canada for further review and 
analysis. The Insurance Bureau of Canada may in 
turn forward the case to FSCO. 

FSCO's Investigations Unit, which comprises 
nine investigators who are primarily former police 
officers, is responsible for investigating all financial- 
services companies and individuals regulated by 
FSCO and not just automobile insurers. As a result, 
the unit's investigation of fraud against individual 
auto insurance companies is not its primary activity. 
FSCO relies on the Insurance Bureau of Canada or 
insurance companies to provide the information 
and evidence necessary to launch an auto insurance 
fraud investigation and win a successful conviction. 
FSCO itself has no jurisdictional authority to pros- 
ecute fraud under the Criminal Code; that author- 
ity belongs to the Ministry of the Attorney General. 
FSCO does have the authority under the Insurance 
Act to prosecute provincial offences such as health- 
care fraud in the auto insurance sector through the 
Provincial Offences Act. It may take action on any of 
the following offences: 

• charging for services not provided; 

• charging, paying, and/or accepting referral 
fees; and 

• making a false or misleading statement to an 
insurer in order to obtain payment for goods 
and services. 

Fines range from a maximum $100,000 on a 
first conviction to a maximum $200,000 for subse- 
quent convictions. FSCO investigators have limited 
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ability to collect information from a health-care 
professional or clinic owner. Therefore, FSCO relies 
on insurance companies to provide the evidence 
that is needed to prosecute. In contrast, FSCO has 
significant authority over insurance companies, 
which are required by law to furnish FSCO with 
full information. FSCO advised us that because the 
burden of proof is high and its investigative powers 
are limited, its chances of winning a prosecution 
against a clinic owner are relatively low. 

We noted that despite the recent increase in 
public awareness of health-care fraud, there has 
been no increase in the number of cases being 
forwarded to FSCO. FSCO received 16 complaints 
against health-care professionals and clinic owners 
between 2008 and the first half of 2011 but had 
obtained convictions only against three health-care 
clinic owners between 2007 and 2010, resulting in 
fines totalling $202,000. 

More recently, insurance companies have begun 
to deal with fraud in civil rather than criminal 
court. In 2010, several insurers sued selected health 
clinics over alleged fraud related to auto insurance 
claims. One insurance company alleged it paid out 
at least $1.2 million to three clinics owned by the 
same individual for medical services that were never 
provided. Other legal action alleged that invoices 
were submitted from health-care clinics totalling 
over $1 million for treatment allegedly provided by 
persons who did not work at the clinic or who had 
left prior to the treatment being billed. At the time 
of our audits, these suits, some seeking restitution 
for several million dollars, were still pending. 

Anti-fraud Measures outside Ontario 

The Insurance Bureau of Canada issued a report 
in February 2011 on "Preventing Auto Insurance 
Fraud in Ontario" to the Ontario Minister of 
Finance. In it, the Insurance Bureau concluded that 
fraud is a "serious" problem in Ontario and recom- 
mended several measures to help address the issue 
and reduce claims costs. We noted that the majority 
of these recommendations reflected actions taken 



by U.S. jurisdictions over the past decade to curb 
fraud. The recommendations included: 

• establishment of a bureau of insurance fraud 
investigations and prosecutions under the 
Insurance Act that would be administered by 
FSCO; 

• increased criminal and civil penalties for 
fraud; 

• civil immunity for persons or organizations 
reporting suspicious activity; 

• mandatory notification of health-care fraud 
convictions to relevant professional colleges 
and the Ministry of Health and Long-Term 
Care; 

• creation of a joint Ontario government-insur- 
ance industry fund to finance fraud investiga- 
tions and prosecutions, and to provide cash 
rewards to people providing information 
leading to a conviction of insurance fraud; 

• mandatory criminal background checks for 
any director, officer, or owner of an independ- 
ent health clinic before granting a license to 
operate; and 

• establishment of a public-education campaign 
on insurance fraud. 

All 50 U.S. states have enacted laws defining 
insurance fraud as a specific crime, and 41 have 
established Insurance Fraud Bureaus. Insurers 
in these jurisdictions must comply with fraud- 
reporting requirements before regulators will 
consider their applications for premium increases. 
Twenty states require insurers to forward all suspi- 
cious claims to a state Insurance Fraud Bureau. 
Other anti-fraud measures taken by one or more 
U.S. states include: 

• rewards of up to $25,000 for information 
about fraudulent acts; 

• public education and advertising campaigns 
such as Virginia's fraud awareness website 
www.stampoutfraud.com and Pennsylva- 
nia's www.helpstopfraud.org. 

• a requirement that accident reports list all 
passengers involved in an accident and not 
just the driver; and 
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• withholding benefits from anyone convicted 
of insurance fraud. 

We found in our discussions with two govern- 
ment-operated insurance corporations in other 
provinces that their monopoly offered several 
distinct advantages in the fight against fraud, 
including: 

• their ability to publish an annual top-10 list of 
auto insurance frauds in their province; 

• operation of a special unit composed of 
former police officers to investigate alleged 
fraudulent claims, along with funding for 
Crown prosecutors dedicated to handling 
insurance fraud; and 

• employment of extensive data-mining tech- 
niques and fraud analytics of claims data to 
identify potential fraud. Each corporation 
maintains a central database of all claims in 
the province, making it possible to identify 
unusual claims or trends that require further 
investigation. 

FSCO, by contrast, is a regulator rather than an 
insurer and thus has no first-hand knowledge of 
auto insurance fraud in this province. Information 
about the occurrence and extent of fraud in the 
auto insurance sector is proprietary information 
belonging to insurance companies. Insurers in 
Ontario have historically been reluctant to acknow- 
ledge publicly any incidences of fraud, or to share 
this information with government organizations, 
including FSCO. Most of the recommendations in 
the Insurance Bureau of Canada's report are beyond 
FSCO's ability to implement without government 
approval. In its 2011 Budget, the government 
announced measures to address auto insurance 
fraud. One measure included the establishment of 
an auto insurance anti-fraud task force. Task-force 
members were appointed in July 2011 with a dead- 
line to issue a final report with recommendations by 
fall 2012. In addition, the government announced 
the recently-created Health Claims for Auto Insur- 
ance (HCAI) system will be used to detect potential 
fraud. FSCO and insurance companies established 
HCAI on February 1, 2011, an online database and 



billing portal requiring health-care providers to 
submit billings for injury claims centrally before 
they are forwarded to insurers for payment. 



RECOMMENDATION 2 



To reduce the number of fraudulent claims in 
Ontario's auto insurance industry and thereby 
protect the public from unduly high insurance 
premiums, the Financial Services Commission 
of Ontario (FSCO) should use its regulatory and 
oversight powers to: 

• help identify potential measures to combat 
fraud, including those recommended by 
the Insurance Bureau of Canada and those 
in effect in other jurisdictions, assess their 
applicability and relevance to Ontario, 
and, when appropriate, provide advice and 
assistance to the government for their timely 
implementation; and 

• ensure development as soon as possible of an 
overall anti-fraud strategy that spells out the 
roles and responsibilities of all stakehold- 
ers — the government, FSCO, and insurance 
companies — in combatting auto insurance 
fraud. 




FSCO shares the Auditor General's concerns 
about fraudulent auto insurance claims. The 
Ministry of Finance's Auto Insurance Anti-Fraud 
Task Force will identify measures to combat 
fraud. FSCO supports and is working with the 
Task Force's steering committee and working 
groups. FSCO will implement any changes in 
regulatory responsibilities arising from the Task 
Force's recommendations. 



RATES FILINGS AND APPROVALS 

All automobile insurers are required under the Insur- 
ance Act and the Auto Insurance Rate Stabilization 
Act to obtain approval from FSCO's Superintendent 
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of Financial Services for the premiums they charge 
and for any changes to the authorized rates. The 
Superintendent is required to reject any application 
where rates: 

• are not just and reasonable in the circum- 
stances; or 

• would impair the financial solvency of the 
insurer; or 

• are excessive in relation to the financial cir- 
cumstances of the insurer. 

Proposed premium rate changes by insurers are 
ultimately a business decision based on factors that 
include past and anticipated claims costs, operating 
costs, and profit levels. Insurance companies are not 
required to submit rate applications at any specific 
interval; instead, they submit when they determine 
that an adjustment, increase, or decrease is appro- 
priate. The main type of filing, a major filing for 
private passenger auto insurance, must be certified 
by a qualified actuary, a business professional who 
uses mathematics to provide expert assessments 
of the financial impact of risk and uncertainty as 
it relates to insurance premiums, expected claims, 
and reserves. 

Approval of Rates 

In order to determine whether the proposed rate is 
justified, FSCO conducts its own actuarial reviews 
using benchmark assumptions. FSCO informed 
us that, in so doing, it recognizes that actuaries 
use a degree of acceptable professional judgment 
in determining assumptions in their assessments 
and may come to different conclusions. FSCO 
also considers other factors, such as the actuaries' 
assumptions that cause differences, rate stability 
for consumers, and the actual rates charged in the 
market compared to other insurers, in determining 
whether the proposed rate is justified and reason- 
able. As a result, FSCO may approve an insurer's 
proposed rate increase even if it is up to three 
percentage points higher than FSCO's calculated 
rate. During the audit, we noted that this practice 
of permitting a three-percentage-point margin was 



not documented in FSCO's filing policies, although 
it subsequently added this practice to its rates 
approval policies when we brought this to FSCO 
staff's attention. 

Between 2006 and 2010, FSCO reviewed and 
approved 293 major filings, as follows: 

• approval of the full request in 65% of filings 
submitted by automobile insurers; 

• approval of a lower-than-requested rate in 
25% of filings; and 

• approval of a higher-than-initially-requested 
rate in 10% of filings. 

It is important that FSCO be consistent in its 
granting of approvals; otherwise, it may provide a 
competitive advantage to one insurer over another 
or may be seen as providing unequal treatment to 
companies and consumers. It is also important, par- 
ticularly when FSCO's conclusions are significantly 
different from those of the insurers' actuaries, 
that FSCO clearly document the rationale for its 
decisions in order to demonstrate fairness and con- 
sistency. We noted that for approvals granted for a 
lower-than-requested rate, in some instances the 
rate approved still exceeded FSCO's calculated rate 
by more than 3% and the reasons for the approvals 
were not adequately documented. In one case, the 
file did not clearly indicate why an insurer received 
permission for an increase that was eight percent- 
age points higher than indicated by FSCO's own 
actuarial determination. In this case, we estimate 
that the additional percentage increase allowed 
could result in additional annual premium income 
of $25 million for the insurer. 

In the cases where FSCO authorized a higher- 
than-initially-requested rate, we also generally 
found inadequate documentation to justify FSCO's 
decision to grant a higher-than-initially-requested 
increase. For example, we were informed that FSCO 
approved a rate higher than initially proposed by 
an insurer on grounds that the insurer had or could 
have financial solvency issues, and it was important 
to protect the company's clients over the long term 
by granting a higher premium than had initially 
been requested. However, we noted that FSCO 
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requested the insurer to increase its rates even 
though there were no financial solvency concerns 
identified by the Office of the Superintendent of 
Financial Institutions, a federal regulator. The 
insurer agreed to the request. FSCO had no formal 
policy on approving the rates of companies with 
financial solvency concerns or on providing guid- 
ance on if and when companies should be asked to 
resubmit their filings for a higher rate increase than 
initially requested. 

We acknowledge, however, that according to 
FSCO, no auto insurance companies operating 
in Ontario have declared bankruptcy since 2002 
or defaulted for financial reasons on their claims 
payouts. 

Review of the Profit Provision 

When determining whether to approve a rate filing, 
FSCO conducts its assessment by factoring in a rea- 
sonable profit for the insurance company based on 
a 12% return on equity (ROE). A study conducted 
in 1988 set the ROE at 12.5% based on its relation- 
ship to the long-term Canada Bond rate, which was 
10% at the time. The ROE was last changed to 12% 
in 1996, and we were advised that FSCO has not 
since conducted a comprehensive review of what 
it considers a reasonable profit for insurance com- 
panies operating in Ontario. Given that long-term 
Canada Bond interest rates were substantially lower 
at the time of our audit, standing at about 3%, have 
been low for some time, and are forecasted to stay 
low for some time, the current 12% ROE could be 
higher than appropriate, assuming that FSCO still 
considers the long-term bond rate to be an appro- 
priate benchmark. In any case, given that it has 
been 15 years since the 12% ROE was established, 
we believe that a reassessment is long overdue. 

Approved Premium Rate Implementation 

To inform consumers of approved premium rate 
changes, FSCO publicly reports on a quarterly basis 
all insurers' rate filing approvals, listing the overall 



average percentage rate change to the authorized 
rates. Consumers renewing their auto insurance 
at the same time might attempt to compare their 
actual rate change to their insurer's approved rate 
change as published by FSCO, but it is unlikely that 
the overall average approved rate change would 
be exactly the same because premiums also reflect 
such variables as the claims experience of the group 
classification and location. As a result, consumers 
are unsure if the new rate they are paying is in 
keeping with that insurer's overall rate approval. 

Consumers can complain to FSCO if they are 
paying an incorrect rate, and FSCO will follow up 
with a review of the complaint and the approved 
rates on file. An investigation will take place 
where warranted. FSCO informed us that between 
the 2005/06 and 2009/10 fiscal years, only five 
of the 22 incorrect rate application cases that it 
investigated were initiated by the public and other 
sources, while the remaining 17 were self -reported 
by insurers. We were advised that when FSCO 
establishes that there has been an error, it fol- 
lows up to ensure that the consumer has received 
a refund and any applicable interest, and may 
conduct an on-site review of the insurer to assess 
procedures and the accuracy of approved rates. 

In the four-year period from 2005/06 through 
2009/10, FSCO levied four fines against insurance 
companies totalling approximately $250,000 for 
rate errors. Such errors can have a significant finan- 
cial impact on consumers — we noted examples of 
overtoiling that totalled between $1 million and 
$11 million. 

For all rate approvals, FSCO requires insurers to 
update their rate manuals and provide FSCO with 
a certificate signed by a senior officer attesting that 
they will charge the approved rates and change 
their information systems accordingly. However, 
FSCO did not have any procedures for periodically 
checking that insurers were charging the approved 
rates. FSCO had not considered the option of requir- 
ing insurance companies to provide attestations 
from third parties, such as their auditors, that the 
approved rates were actually being applied correctly. 
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RECOMMENDATION 3 



To ensure that the Financial Services Commission 
of Ontario (FSCO) fairly and consistently author- 
izes auto insurance company premium rate chan- 
ges while protecting consumers, FSCO should: 

• update and document its policies and pro- 
cedures for making rate decisions — particu- 
larly for applications that differ from its own 
assessments — and for properly assessing rate 
changes in light of actual financial solvency 
concerns of insurance companies; 

• review what constitutes a reasonable profit 
margin for insurance companies when 
approving rate changes, and periodically 
revise its current assessment to reflect sig- 
nificant changes; and 

• establish processes for verifying or obtaining 
assurance that insurers actually charge only 
the authorized rates. 




FSCO operates one of the most robust premium 
rate approval processes in North America and 
fully supports further strengthening of its pro- 
cess. In particular, FSCO acknowledges the need 
to update policies and procedures to support 
decisions regarding rate filings. 

As part of deciding if rates are "just and 
reasonable," FSCO determines whether the rate 
charged is adequate to cover all claims costs and 
expenses. In addition, case law requires a bal- 
ancing of interests in the interpretation of "just 
and reasonable." 

Last year, FSCO also identified the need to 
complete a review of an appropriate profit provi- 
sion. It will finalize the process and retain a con- 
sultant to provide expert analysis on this issue. 

FSCO ensures that consumers are issued 
refunds where an insurer has charged the 
incorrect rate. FSCO plans to enhance its cur- 
rent processes for verifying or obtaining assur- 
ances from insurers that they are charging only 
authorized rates. 



DISPUTE RESOLUTION SERVICES 

According to FSCO, the mandate of its Dispute 
Resolution Services Branch is to provide a fair, 
timely, accessible, and cost-effective process for 
resolving claimant disputes with insurers involving 
the entitlement to and/or the amount of statutory 
accident benefits. Common examples of disputes 
mediated include those in which applicants 
seek greater medical benefits, higher income- 
replacement compensation, more in housekeeping 
and/or home-maintenance costs, or attendant care 
benefits. 

Mediation through FSCO is a legislated manda- 
tory first step under the Insurance Act, and neither 
party can proceed to FSCO's arbitration process 
or court unless mediation occurs first. Mediation 
services are free for consumers, but the insurance 
companies pay $500 for each hearing. 

The Insurance Act requires that mediation be 
completed within 60 days of the filing of the appli- 
cation unless both parties agree to an extension. 
FSCO's internal service standards require that a 
mediation application be assigned to a mediator 
within three weeks of receipt and that a mediator 
file within seven days following the mediation 
process a report that lists issues settled and any that 
remain in dispute. These services are intended to 
help insurance companies and claimants resolve 
disputes quickly and cost-effectively and to ensure 
that entitled claimants receive any medical benefits 
and compensation owing within a reasonable time. 

We found that FSCO was unable to meet its 
service standards due to the large volumes of 
mediation applications filed and its limited staff 
resources. In the 2010/11 fiscal year, no mediations 
were completed within 60 days of filing, and most 
applications were dealt with between 10 and 12 
months after the date of filing. It also took approxi- 
mately 15 weeks — instead of three — to assign an 
application to a mediator. However, once the medi- 
ation process was completed, the mediators met the 
requirement of issuing a report within seven days in 
95% of cases. 
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FSCO attributed the delays to the dramatic 
increase in mediation applications over the last 
five fiscal years. Figure 7 shows that the number 
of mediation case hearings increased by 135% 
during this period, while the number of mediation 
applications pending increased by 645%. FSCO 
informed us that it expected the mediation backlog 
would continue to increase, because it was pro- 
jecting more than 36,000 new applications in the 
20011/12 fiscal year, up 18% from 2010/11. 

FSCO has implemented measures over the last 
five years to improve its productivity and help 
address the growing backlog of mediation applica- 
tions, and it is now completing 50% more cases 
with no staff increases. In addition, changes to the 
regulations under the Act imposed a new require- 
ment in September 2010 stipulating that an appli- 
cant may not file for mediation if he or she failed 
to attend an insurer's medical assessment (40% of 
applicants have historically failed to attend such 
assessments). 

At the end of our fieldwork, FSCO informed us 
that it was seeking approval to hire external media- 
tion service providers to supplement its own work- 
force and help address the existing and anticipated 
backlog caused by government hiring restrictions 
which do not allow it to take on more staff. 

The current rate of injury claims that result in 
mediation stands at about 50% of all claims. We 
believe that this high rate could indicate significant 
dissatisfaction by claimants with the handling 
of claims by insurers and/or lack of clarity from 
FSCO in the guidance and manner in which statu- 
tory accident benefits are administered. It could 
also suggest, in part, that a burgeoning industry 
providing legal consulting services to claimants 
is encouraging them to challenge insurers for 
increased benefits and compensation through the 
mediation process. This maybe the case particu- 
larly in the Greater Toronto Area, where about 80% 
of all mediation applications originate, even though 
only 45% of automobile accidents involving injuries 
occur in the GTA. 



Figure 7: Growth in Mediation Applications, 
2006/07-2010/11 

Source of data: Financial Services Commission of Ontario 
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FSCO and the insurance companies we spoke to 
cited several factors they said led to the increasing 
demand for mediation, including over-utilization of 
benefits, the impact of recent legislative changes, 
people seeking more compensation during tough 
economic times, and the fact that 99% of claimants 
who dispute their insurer's decision about their 
claim use a legal service and seek monetary settle- 
ments instead of health-care and support benefits. 
It is also possible that insurance companies are 
being tougher in assessing claims as they respond 
to their growing incurred loss ratios and declining 
revenues from interest-bearing investments during 
this recent economic downturn, and to pressure 
from FSCO on insurers to fight fraud. 

The actual reasons for the higher number 
of mediation cases cannot be determined from 
the information FSCO captures. Although FSCO 
captures mediation details in individual reports, 
there is no attempt to evaluate and summarize this 
information because it is considered to be confiden- 
tial. Therefore, FSCO does not regularly assess the 
nature of the disputes, the initial positions of the 
parties, the details of solutions to resolved matters, 
and the details of those that were not resolved. This 
information would help FSCO identify matters of 
frequent dispute and systemic issues. 

We found that FSCO tracks the disputed issues 
at mediation only by benefit type. From 2006 to 
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will look at additional data collection that might 
assist in identifying ways to reduce the high 
demand for dispute resolution services. 

FSCO has implemented a number of meas- 
ures and initiatives that have increased pro- 
ductivity and has managed to close 50% more 
files during the last five years. Since completion 
of the audit field work, additional initiatives 
have been developed and will be implemented 
through the fall and winter. FSCO has engaged 
the Ministry of Finance in developing an action 
plan to address the backlog. 



2010, FSCO's records indicated that the top issues 
in dispute were medical benefits, cost of examina- 
tions, housekeeping and/or home maintenance, 
attendant care, and income replacement. However, 
this information is not sufficiently detailed to 
permit an investigation of the root causes of cases 
that go to mediation. We were advised that FSCO 
consulted with its mediators on possible improve- 
ments to the system, policies, and guidelines to 
reduce backlogs in 2007 and 2009, but no regular 
process existed at the time of our audit to obtain 
mediators' opinions on possible systemic problems 
and possible improvements and clarifications to 
SABS guidelines and policies to reduce the demand 
for mediation. 



RECOMMENDATION 4 



To ensure that the Financial Services Commis- 
sion of Ontario meets its mandate to provide 
fair, timely, accessible, and cost-effective 
processes for resolving disputes over statutory 
accident benefits, it should: 

• improve its information-gathering to help 
explain why almost half of all injury claim- 
ants seek mediation, as well as how disputes 
are resolved, and to identify possible sys- 
temic problems with its SABS benefits poli- 
cies that can be changed or clarified to help 
prevent disputes; and 

• establish an action plan and timetable for 
reducing its current and growing backlog to 
a point where it can provide mediation ser- 
vices in a timely manner in accordance with 
legislation and established service standards. 

| FSCO RESPONSE 

FSCO captures information about disputes 
submitted for mediation, collects aggregate 
statistical information, and compiles reports on 
profiles of applications received, types of bene- 
fits mediated, workload analysis, processing 
time, and whether mediation fully or partially 
settled disputes, or failed to settle them. FSCO 



PERFORMANCE MEASURES 

In its annual business plan submitted to the Minis- 
ter of Finance, FSCO established three performance 
measures for its regulatory responsibilities over the 
auto insurance industry, as follows: 

• average number of days taken to approve pri- 
vate automobile premium-rate applications, 
compared to its target of 45 days; 

• percentage of mediation reports completed 
within seven days of conclusion of mediation, 
compared to its target of 94%; and 

• weighted ratio of administrative costs to dol- 
lars paid out of the Motor Vehicle Accident 
Claims Fund, compared to its target of 28%. 

In the five fiscal years ending in 2010/11, 
FSCO generally met these publicly stated targets. 
However, in our view, these targets do not report 
on its success in protecting the public interest with 
respect to auto insurance or provide useful insight 
into its regulatory oversight responsibilities and 
activities. As well, there are no performance targets 
regarding the financial health of insurance compan- 
ies. In particular, the targets include no benchmark- 
ing of the cost-effectiveness of auto insurance in 
Ontario. In addition, the target established for 
mediation services does not reflect the overall 
timeliness of service levels. As discussed in a previ- 
ous section, FSCO generally meets the seven-day 
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target for issuing a report following a mediation. 
However, it is of greater importance to consumers 
to note that FSCO takes between 10 to 12 months to 
complete a mediation hearing once an application 
has been received, instead of the legislated require- 
ment of 60 days. 

The Financial Services Commission of Ontario 
Act requires FSCO to file its annual report within a 
reasonable time after the close of each fiscal year 
to the Minister, who then tables it in the Legisla- 
tive Assembly. As of July 2011, however, FSCO's 
annual report for the year ending March 31, 2010, 
had not been tabled by the Minister of Finance and 
thus had not been made public. We also noted that 
FSCO does not report publicly on its performance. 
For example, it does not make public its annual 
business plan, and its latest annual report does not 
include objective and outcome-based performance 
measures, targets, or details about its accomplish- 
ments in meeting stated goals and targets. 

We did note, however, that FSCO does make 
public its Statement of Priorities as required under 
the Financial Services Commission of Ontario Act. In 
it, FSCO sets out its proposed priorities and initia- 
tives to meet changing economic and marketplace 
conditions in the coming year as well as its accom- 
plishments from the previous year. 



RECOMMENDATION 5 



In order to provide the public, consumers, 
stakeholders, and insurers with meaningful 
information on its auto insurance oversight 
and regulatory activities, the Financial Services 
Commission of Ontario should report timely 
information on its performance, including 
outcome-based measures and targets that more 
appropriately represent its key regulatory activ- 
ities and results. 

FSCO RESPONSE 

FSCO agrees that the public, consumers, and 
stakeholders should be provided with more 



meaningful information on its performance in 
the oversight of the auto insurance system. In 
its 2011 Statement of Priorities, published in 
June 2011, FSCO indicated that it will develop 
improved performance measures and establish 
standards against which it can be judged in all 
of the sectors it regulates. The existing measures 
will be reviewed and updated. 



MOTOR VEHICLE ACCIDENT CLAIMS 
FUND UNFUNDED LIABILITY 

The Motor Vehicle Accident Claims Fund (Fund) 
is generally considered the "payer of last resort." 
It compensates victims of automobile accidents 
caused by uninsured motorists, drivers of stolen 
vehicles, or hit-and-run drivers, when no other 
automobile or liability insurance is available to pay 
a claim. Victims can apply to the Fund, which pays 
statutory accident benefits and any tort judgments. 
The Fund operates under the authority of the Motor 
Vehicle Accident Claims Act and is administered by 
FSCO. The Fund also contracts with an independ- 
ent adjuster to investigate claims and handle statu- 
tory accident benefit claims payments. Payments 
by the Fund rose from $17.9 million for 553 claims 
in the 2006/07 fiscal year to $21 million for 585 
claims in 2010/11. 

According to FSCO's consulting actuary, as 
of March 31, 2011, the Fund's assets were sub- 
stantially less than what is needed to satisfy the 
estimated lifetime costs of all claims currently in 
the system, resulting in an unfunded liability. As 
Figure 8 indicates, the Fund's unfunded liability 
was $109 million as of March 31, 2011, but FSCO 
forecasts that it will grow to $323 million by the 
2021/22 fiscal year unless the Fund receives signifi- 
cant additional revenue. 

The Fund is supported primarily by a fee on the 
issuance or renewal of each Ontario driver's licence, 
which works out to be $15 paid every five years. In 
2010/11, the Fund received $28.7 million in fees. 
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Figure 8: Motor Vehicle Accident Claims Fund Actual and Projected Unfunded Liability, 2000/01-2021/22 
($ million) 



Source of data: Financial Services Commission of Ontario 
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The fee was last increased in September 2004 by 
$10 on a driver's licence five-year renewal. Our 
discussion with management noted that there is no 
plan or timetable in place to eliminate the unfunded 
liability in a reasonable amount of time. We estimate 
that the Fund would need an additional $30 million 
per year — that is, double the current annual fee rev- 
enue — for the next 10 years to eliminate the existing 
and projected unfunded liability. This could require 
FSCO to seek Ministry of Finance approval for doub- 
ling the current $15 driver's licence renewal fee. 



RECOMMENDATION 6 



To ensure that the Motor Vehicle Accident 
Claims Fund (Fund) is sustainable over the 
long term and able to meet its future financial 
obligations, the Financial Services Commis- 



sion of Ontario should establish a strategy and 
timetable for eliminating the Fund's growing 
unfunded liability over a reasonable time period 
and seek government approval to implement 
this plan. 



We acknowledge the Auditor General's findings 
regarding the unfunded liability of the Motor 
Vehicle Accident Claims Fund (Fund). FSCO's 
current 10-year projections suggest that the 
current positive cash balance should adequately 
provide for the Fund's statutory payment obli- 
gations to claimants for at least the next eight 
years through to the 2019/2020 fiscal year. Cash 
flow studies are done annually and the next 
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one will be done in August 2012 to re-assess the 
10-year cash flow projections. 

In the past, the government has taken appro- 
priate and timely steps to address the Fund's 
needs. FSCO will continue its regular engage- 
ment with the Ministry of Finance on the Fund's 
evolving financial status to ensure that statutory 
payment obligations to Fund claimants are met. 



OTHER MATTER 

Assessment of Health-system Costs 

The Insurance Act was amended in 1996 to require 
all automobile insurers operating in Ontario to 
pay an annual "assessment of health-system costs" 
to recover the costs to the province of providing 
medical care to people injured through someone 
else's fault. The government of the day initially set 
the assessment at about $80 million a year for the 
entire industry to help defray costs incurred by the 
Ministry of Health and Long-Term Care that ought 
to be paid by insurers. FSCO is responsible for col- 
lecting the assessment from insurers, with each 
insurer paying a pro-rated share of the total. 

In 2005, our audit of the recovery of health 
costs resulting from accidents led us to conclude 
that the Ministries of Health and Finance did not 
have satisfactory policies and procedures in place 
to monitor the adequacy of the initial $80-million 
annual assessment. Subsequently, the government 
increased the annual assessment in September 
2006 to about $142 million. 

The Health and Finance Ministries reported 
in our 2007 follow-up that they had established a 
joint working group that year to conduct further 
analysis to ensure that future assessment amounts 
adequately cover the cost of health care provided 
to individuals injured in automobile accidents. The 
ministries also said at the time that it would take 
some time to develop the appropriate mechanism. 



However, the Ministry of Finance informed us that 
no progress had been made in this area as of July 
2011 and that the government was not considering 
any increase in the assessment. 

We also noted that overall health-care spending 
by the Ministry of Health and Long-Term Care has 
increased by about 25% since the assessment was 
last adjusted in the 2006/07 fiscal year. In addi- 
tion, medically-related SABS benefits costs have 
increased by almost 120% over the same period, 
although some of the medical costs, such as physio- 
therapy and massage therapy, may be unrelated 
since they may not normally be covered by the 
Ministry of Health and Long-Term Care. 

We compared Ontario's assessment of health- 
system costs to those of other jurisdictions and 
found that Ontario's is among the lowest in Canada 
when measured on a per-registered-vehicle basis, 
as illustrated in Figure 9. If Ontario's assessment 
per registered vehicle were raised to the average of 
other provinces, the assessment would increase by 
50%, or about $70 million, to $214 million. Assum- 
ing that the insurance industry was successful in 
passing this cost on to vehicle owners, this change 
would likely add almost $10 to the insurance pre- 
mium for each vehicle in Ontario. 

Figure 9: Provincial Comparison of Assessments of 
Health-system Costs on Auto Insurance Industry, 2011 
($ per registered vehicle) 

Source of data: Office of the Auditor General of Ontario, provincial finance 
ministries, and Public Accounts 
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RECOMMENDATION 7 



In view of the fact that it has been five years 
since the last review of the assessment of health- 
system costs owed by the auto insurance sector 
despite the significant increase in health-care 
costs related to automobile accidents over the 
same period, the Financial Services Commission 
of Ontario should work with the Ministry of 
Finance, the Ministry of Health and Long-Term 
Care, and the insurance industry to review the 
adequacy of the current assessment amount. 



| FSCO RESPONSE 

FSCO agrees with the Auditor General's recom- 
mendation that health-care assessments paid to 
the government by auto insurance companies 
would benefit from more regular review. The 
responsibility for initiating the review rests 
with the government. FSCO will ensure that 
the Ministry of Finance is aware of the auditor's 
recommendation and will support the Ministry 
of Finance in any future review as requested. 



Chapter 3 



Ontario Energy Board 



Electricity Sector- 
Regulatory Oversight 




Electricity is an essential commodity required for 
the well-being of Ontario's economy and the day- 
to-day activities of its citizens. That, along with 
the electricity sector's status as a near-monopoly 
necessitated a system of oversight and regulation 
to ensure sustainability and cost-effectiveness in 
the generation and delivery of electricity to meet 
the needs of consumers, business, and industry. 
Ontario's electricity sector serves 4.7 million cus- 
tomers and is composed of several key entities, as 
illustrated in Figure 1. 

The Ontario Energy Board (Board) was origin- 
ally established in 1960 to set rates for the sale 
and storage of natural gas and to approve pipeline 
construction projects. Over time, its powers and 
responsibilities evolved through legislation. In 1973, 
it became responsible for reviewing and reporting to 
the Minister of Energy on electricity rates charged 
by the old Ontario Hydro, a function that it per- 
formed until the late 1990s, when Ontario Hydro 
was split into several successor companies. 

Today, the Board still regulates the province's 
natural-gas sector, but devotes most of its time to 
oversight of the electricity sector in Ontario. The 
Board is required to oversee the sector through 
effective, fair, and transparent processes, in accord- 



ance with the Electricity Act, 1998 and the Ontario 
Energy Board Act, 1998. The objectives of the Board 
include protecting the interests of consumers, 
facilitating the maintenance of a financially viable 
electricity sector, and promoting efficiency and cost- 
effectiveness in the sector. The Board's key functions 
with respect to fulfilling these objectives include: 

• setting prices for electricity and its delivery; 

• monitoring electricity markets and licensing 
participants; 

• approving the annual expenditure and 
revenue requirements of the Ontario Power 
Authority and the Independent Electricity 
System Operator; and 

• reviewing and setting regulatory policies. 
The Lieutenant Governor-in-Council appoints 

members to the Board. At the time of our fieldwork, 
the Board had eight members — seven full-time and 
one part-time — supported by a staff of about 170. 
Board operating costs were $34.8 million in the 
2010/11 fiscal year, with 80% of that paid by regu- 
lated electricity entities and 20% by the natural-gas 
sector. 



Audit Objective and Scope 



The objective of our audit was to assess whether 
the Ontario Energy Board (Board) had adequate 
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Figure 1: Selected Key Roles of Entities in Ontario's Electricity Sector 

Prepared by the Office of the Auditor General of Ontario 



SECTOR CO-ORDINATION, OVERSIGHT, AND REGULATION 



• Ministry of Energy: sets overall policy and legislative framework 

• Ontario Power Authority: prepares overall plan and procures power supply 

• Ontario Energy Board: sets and regulates some electricity prices and performs 
other regulatory activities 



ELECTRICITY SUPPLY 

Generated by: Transmitted through: Distributed by: 

• Ontario Power Generation ^^^^^^ • Transmission lines operated ^^^^^ > • Hydro One 

• Private companies mainly by Hydro One • 79 local utility companies 
(e.g., Bruce Power) (e.g., Toronto Hydro) 



ELECTRICITY SYSTEM/ MARKET 

Managed and operated by: 

• Independent Electricity System Operator 



systems and procedures in place to protect the 
interests of electricity consumers and ensure that 
the electricity sector provides reliable and sustain- 
able energy at a reasonable cost. 

A secondary and equally important objective of 
our report was to look at the regulatory context of 
the charges on Ontario electricity bills and explain 
what these charges relate to. In keeping with our 
aim to inform readers in the simplest terms pos- 
sible, we use the terms "ratepayer," "customer," and 
"consumer" interchangeably in this audit report. 

The scope of our work included a review and 
analysis of rate applications and filing guidelines 
and interviews with members and appropriate 
staff at the Board. We also met with staff from 
other provincial agencies, including the Ministry of 
Energy, the Ontario Power Authority, the Independ- 



ent Electricity System Operator, Ontario Power 
Generation, and Hydro One. 

We also spoke with various participants and 
stakeholders in the electricity market, including 
local distribution companies and intervenors, to 
get their perspective on their interactions with 
the Board as well as its regulatory processes. 
Intervenors are individuals or groups representing 
consumers or other interested parties who actively 
advocate on their behalf in the hearing processes. 
In addition, we researched the operations of electri- 
city regulators in other Canadian jurisdictions and 
engaged an independent consultant with expert 
knowledge of electricity regulation across Canada 
to assist us on an advisory basis. The Board follows 
a quasi-judicial process to make its rate-setting 
decisions. These decisions and the judgment of the 
Board panels were not a subject of this audit. 



Electricity Sector— Regulatory Oversight 



Before beginning our work, we developed audit 
criteria that we used to achieve our audit objective. 
These were discussed with and agreed to by the 
Board's senior management. 




A key role of the Ontario Energy Board (Board) 
as regulator of the electricity sector is to protect 
consumers while providing a reasonable rate of 
return for the industry by setting just and reason- 
able prices. This role is especially important given 
that electricity prices for the average consumer 
have increased 65% since the restructuring of the 
electricity sector in 1999, and prices are expected to 
rise another 46% in the next five years. 

We observed that Board staff undertook to pro- 
vide Board members with useful analyses and other 
information to assist them in their deliberations. 
As well, the Board has undertaken a number of 
initiatives to educate consumers about the charges 
on their electricity bills, including an on-line bill 
calculator that has garnered industry recognition. 
However, we identified certain factors that could 
limit the Board's ability to perform its regulatory 
duties to the extent that consumers and the elec- 
tricity sector might reasonably expect. Among our 
observations: 

• The Board is not responsible for ensuring that 
electricity bills as a whole are just and reason- 
able, insofar as its jurisdiction extends to only 
about half of the total charges on a typical 
bill. The Board's role is largely limited to set- 
ting rates for the nuclear power and some of 
the hydro power produced by Ontario Power 
Generation (OPG), along with transmission, 
distribution, and certain other charges. The 
other half of power bills is based on govern- 
ment policy decisions over which the Board 
has no say. For example: 
• About 50% of the electricity sold to resi- 
dential customers comes from suppliers 



who signed long-term contracts with the 
government or the Ontario Power Author- 
ity, and the price of this power accounts for 
65% of the cost of the electricity compon- 
ent on the typical bill. However, the Board 
has no regulatory oversight role with 
respect to this portion of the electricity 
charge. Rather, it regulates only electricity 
from certain OPG nuclear and hydro plants, 
which constitutes about one-third of the 
electricity charges on a typical bill. 

• The debt retirement charge that consumers 
pay each month was originally created by 
the government in 1999 to help pay off 
the estimated "residual stranded debt" of 
$7.8 billion that remained after the old 
Ontario Hydro was broken up. The Board 
has no oversight role with respect to this 
charge or how long it is to be applied to 
consumers' electricity bills. 

• The Board has regulatory oversight over 
only about $190 million of the close to 
$900 million collected from ratepayers 
to administer and operate the electricity 
market and to meet other legislated 
requirements. 

• In areas where it does have jurisdiction, the 
Board sets rates using a quasi-judicial process 
that requires utilities and other regulated 
entities, such as OPG and Hydro One, to 
justify any proposed rate increases in a public 
hearing. Many small and mid-sized utilities 
said that this process costs ratepayers an aver- 
age of between $100,000 and $250,000 per 
application — or as much as half the revenue 
increase sought in the first place by these 
utilities. These costs are generally incurred 
once every four years and are recovered from 
consumers over the next four-year period. 

• Individuals or organizations wishing to par- 
ticipate in the hearings on behalf of consumers 
can obtain intervenor status, and can qualify 
for reimbursement of their expenses by util- 
ities and other regulated entities. However, 
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Detailed Audit Observations 



many of these utilities and other regulated 
entities cited the high cost of providing the 
large quantities of detailed information 
requested by intervenors and called for better 
co-ordination by the Board to manage these 
requests. 

• In monitoring utilities for compliance with 
its guidelines and reporting requirements, 
the Board identified a number of significant 
deficiencies in the utilities' record-keeping 
and reporting practices. This could be an 
indication of inaccuracies in the information 
the Board uses to make decisions. However, 
the Board does not consistently follow up to 
ensure that the noted deficiencies were cor- 
rected by the utilities. 

• Consumers can purchase their electricity 
either through their utility at the Regulated 
Price Plan prices set by the Board or through 
an electricity retailer at a price set by the 
retailer. Some 15% of residential customers, 
looking for price protection and stability 

on their power bills, signed fixed-price con- 
tracts with electricity retailers. However, we 
found that these consumers could be paying 
anywhere from 35% to 65% more for their 
electricity than they would pay had they not 
signed those contracts. In the last five years, 
the Board has received more than 17,000 
complaints from the public; the overwhelming 
majority of them have been against electricity 
retailers. Issues included misrepresentation 
by sales agents and even forgery of signatures 
on the contracts. Although the Board follows 
up on complaints, the number of enforcement 
actions taken against retailers has been very 
limited. 

• The Board has a well-structured performance- 
reporting process, but its performance meas- 
ures need to be more results-based rather 
than process-oriented. 



OVERVIEW OFTHE ONTARIO ENERGY 
BOARD AND THE ELECTRICITY SECTOR 

The Ontario Energy Board (Board) was founded in 
1960 to regulate the natural-gas sector in Ontario. 
In 1973, its role was expanded to include the 
electricity sector. A significant shift in the Board's 
mandate came when the government enacted the 
Energy Competition Act, 1998 (Act), which broke 
up the old Ontario Hydro into several successor 
companies and sought to introduce competition to 
the electricity sector. 

The Act mandated the Board to protect the inter- 
ests of consumers while simultaneously ensuring a 
financially viable electricity industry. More detail 
about legislative and policy changes since 1999, 
and the impact of these changes on the electricity 
sector and the Board, is shown in Figure 2. 

IMPACT ON CONSUMERS 

Ontario consumers have experienced significant 
electricity-cost increases over the past decade as a 
result of major changes to the province's electricity 
sector. Since 1999, the average residential con- 
sumer using 800 kilowatt hours (kWh) per month 
has seen a 65% increase in his or her power bill. The 
Ministry of Energy predicted in its 2010 Long-term 
Energy Plan that residential electricity bills will rise 
another 46% over the next five years to help pay for 
upgrades to Ontario's existing nuclear and natural- 
gas generation capacity and its transmission and 
distribution facilities, and to help finance new and 
cleaner renewable-energy generation. 

A summary of the impact on energy bills of the 
major policy changes since 1999 is shown in Figure 3. 

UNDERSTANDING ELECTRICITY BILLS 



In 2004, the government passed a regulation 
requiring electricity bills for low- volume consumers 
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Figure 2: Government Legislation and Policy Changes in the Electricity Sector, 1998-2011 

Prepared by the Office of trie Auditor General of Ontario 





Energy Competition Act, 1998 


• Breaks up Ontario Hydro into several companies 

• Ontario Energy Board (Board) assumes responsibility for regulating three 
Ontario Hydro successor companies and local distribution companies 


Electricity Pricing, Conservation and Supply 


• Caps electricity price at 4.3<t/kWh, for two years, effective May 1, 2002 


Act, 2002 


• Freezes transmission and distribution rates until at least May 1, 2006 


Ontario Energy Board Consumer Protection 


• Creates a management committee to oversee Board activities 


and Governance Act, 2003 


• Strengthens Board powers to protect and educate consumers 


Ontario Energy Board Amendment Act 


• Replaces 4.3<t/kWh price cap as of April 1, 2004, with 4.7$/kWh for the 


(Electricity Pricing), 2003 


first 750 kWh/month, and 5.5<t/kWh beyond 750 kWh/month 
• Allows local distribution companies to recoup costs by lifting freeze imposed 
by Electricity Pricing, Conservation and Supply Act, 2002 


Electricity Restructuring Act, 2004 


• Amends Ontario Energy Board Act, 1998 an6 Electricity Act, 1998 

• Board assumes responsibility for Market Surveillance Panel 

• Establishes Ontario Power Authority (OPA) to ensure adequate, reliable, and 
secure electricity supply in Ontario 


Minister's Directive to Board (2004) 


• Develops smart-meter implementation plan 


Minister's Directive to OPA (2006) 


• Develops plan to replace coal-fired generation with cleaner sources as soon 
as possible 


Green Energy and Green Economy Act, 2009 


• Establishes responsibility for Board and other entities to achieve objectives 
of conservation, promotion of renewable energy, and technological 
innovation 


Harmonized Sales Tax (2010) 


• Adds 8% to total electricity bill effective July 1, 2010 


Energy Consumer Protection Act, 2010 


• Requires that Ontarians be provided with the information they need about 
electricity contracts and prices and that consumers be protected by fair 
business practices effective January 1, 2011 



Ontario Clean Energy Benefit (2011) • 10% discount on electricity bill for five years from January 1, 2011 



(residential and small-business consumers) to show 
four categories of charges: Electricity, Delivery, 
Regulatory, and Debt Retirement. The regulation 
also specifies how these categories of charges are 
to be explained on or with the bill. A sample bill 
for an average Toronto Hydro residential consumer 
with an 800 kWh monthly consumption (or about 
830 kWh when adjustment due to loss in the distri- 
bution system is included) is shown in Figure 4. 
The various charges break down as follows: 
• "Electricity" is the cost of the actual power 
consumed, which the province obtains pri- 
marily from Ontario Power Generation (OPG) 
and from suppliers who have signed contracts 
with the government or the Ontario Power 
Authority (OPA). The presentation of this 



charge on bills varies, depending on whether 
the consumer buys from a utility or has signed 
a contract with a retailer. In Ontario, 85% of 
residential consumers purchase their electri- 
city from local utilities and pay what is known 
as Regulated Price Plan (RPP) prices, while 
the remaining 15% purchase their electricity 
from electricity retailers. 

RPP prices are set by the Board. Time-of- 
use RPP prices — where the price of electricity 
varies depending on when during the day 
the consumer uses power — apply if the con- 
sumer's utility has migrated to time-of-use 
billing. Otherwise, two-tiered RPP pricing — 
where the price of electricity varies depending 
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Figure 3: Electricity Costs for Average Toronto Consumer Using 800 kWh of Electricity a Month, 1999-2011 ($) 

Prepared by the Office of trie Auditor General of Ontario 
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on how much power the consumer uses per 
month — applies. 

Consumers with retail contracts pay the 
price stipulated in their contracts plus a 
Global Adjustment — mostly consisting of the 
difference between the market price and the 
price paid to generators as set by the Board for 
OPG or under contract with the government 
or the OPA. The Global Adjustment has been 
rising steadily over the last few years and 
is expected to continue to rise as a result of 
investments in existing generation capacity 
and renewable power generation. The RPP 
prices calculated by the Board include a fore- 
cast of the Global Adjustment. RPP consumers 



therefore do not see a separate Global Adjust- 
ment charge on their electricity bills. 

• "Delivery" is the cost of transmitting and dis- 
tributing electricity from the generator to the 
consumer. Transmission is handled primarily 
by Hydro One over high-voltage wires con- 
necting generators across the province to local 
utilities, which handle distribution to homes 
and businesses. Delivery rates vary across 

the province, with rural and remote locations 
generally paying higher rates. 

• "Regulatory" is the cost to operate the electri- 
city system and maintain the reliability of the 
provincial grid. This includes the operational 
costs of the Independent Electricity System 
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Figure 4: Monthly Electricity Bill Comparison (Regulated Price Plan vs. Retail Contract Consumer) 

Source of data: Ontario Energy Board website, August 2011 
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$127.33 
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(what is this?) 


(-12.73) 


Total Amount 


$114.60 



Operator (IESO) and the OPA, charges to 
partly offset the higher cost of providing 
electricity to rural and/or remote areas, and 
a charge to cover administrative costs of local 
utilities. 

• "Debt Retirement Charge" is mandated by 
the government to help pay off the residual 
stranded debt of the old Ontario Hydro that 
could not be funded by other revenues. This 
charge will be collected from consumers until, 
in the opinion of the Minister of Finance, the 
debt has been eliminated. 

• "Ontario Clean Energy Benefit" is a 10% dis- 
count on the total electricity bill that applies 
for five years starting January 1, 2011, to 
help offset price increases. The annual cost of 
this rebate is estimated at $1.1 billion and is 







Electricity Retail Contract 
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funded by taxpayers through the Ministry of 
Energy's annual appropriation. 

REGULATORY OVERSIGHT OF 
ELECTRICITY 

The Ontario Energy Board (Board) is mandated to 
regulate the electricity sector in Ontario. However, 
its authority to review and regulate is limited to 
only about half the charges on the average residen- 
tial or small-business bill, as illustrated in Figure 5. 

What the Board Does-and Does 
Not- Regulate 

For the electricity component of a bill, the Board 
regulates the cost of power from certain OPG assets 
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such as nuclear and large hydro generating plants; 
however, the costs of power from OPG's other 
generation assets, as well as the costs of electricity 
supplied under contracts negotiated by the OPA 
and under power agreements with non-utility sup- 
pliers, are not subject to Board regulation. Every 
six months, the Board reviews the RPP electricity 
prices being paid by residential and small-business 
consumers and, if necessary, adjusts them to ensure 
that they reflect the cost of supplying electricity to 
those consumers. 

The Board regulates the entire delivery compon- 
ent (that is, all of the transmission and distribution 
charges). 

For the regulatory component, the Board regu- 
lates the operational costs of the IESO and the OPA, 
but there are other regulatory costs that it does not 
regulate. 

The debt retirement charge is not subject to 
Board regulation. 

CHARGES SUBJECTTO REGULATORY 
OVERSIGHT 

The old Ontario Hydro followed a relatively 
straightforward rate-setting process, calculating 
rates on a cost-recovery basis. It was not required 
to consider whether the costs incurred were rea- 
sonable or whether all costs were being billed to 



consumers over an appropriate time period. The 
current system is more complicated. It requires 
that the Board set just and reasonable rates, with 
the result that the Board's information needs are 
more complex than those during the time of the 
old Ontario Hydro. Such rate-setting oversight 
involves assessing projected operating costs as well 
as recovering the cost of capital investments. 

In the case of such infrastructure investments, 
the Board must determine whether these capital 
costs are fairly distributed between current and 
future consumers. It must also examine the costs of 
building or acquiring different types of electricity 
assets, and how long they will last. Regulated 
entities investing in such assets are entitled to a 
reasonable rate of return on their investment, and 
their returns are largely guaranteed once the Board 
approves their rates. For proposed capital invest- 
ments, the Board must satisfy itself that the invest- 
ments are needed. For example, is more investment 
required to maintain or enhance the system's 
reliability, when should new electricity generators 
be connected to the transmission system given 
forecasted future demand, and how should new 
initiatives such as the smart grid be implemented? 

In fulfilling its rate-setting role, the Board fol- 
lows a quasi-judicial process that is open to public 
participation. The Board advised us that it takes 
seriously the need for its adjudication decisions to 



Figure 5: Percentage of Electricity Bill Regulated by the Board, 2010 (average utility customer consuming 
800 kWh a month at a cost of $116) (%) 

Prepared by the Office of the Auditor General of Ontario 
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be made — and to be seen to be made — independ- 
ently and impartially. The hearing process must 
comply with statutory requirements and principles 
of administrative law. 

The regulatory process the Board follows is sum- 
marized in Figure 6. 

Applicants, including utilities, OPG, and Hydro 
One, are expected to provide sufficient detail about 
proposed rate increases to enable the Board to 
determine whether the proposed rates are just and 
reasonable, although the onus is on applicants to 
prove that the proposed increases are justified. In 
considering such applications, the Board examines 
the applicant's forecasts, along with financial and 
operational details, in a public forum. Applicants 
must provide documentation to cover current oper- 
ations and historical data going back three years. 
The Board aims to set rates that allow applicants to 
recover their ongoing operating costs and the cost 



of capital expenditures over an appropriate time 
period and earn a reasonable rate of return. The 
rate of return set by the Board for 2011 was 9.58%. 

Rates and fees subject to regulation include the 
rate charged for power supplied by OPG's nuclear 
and large hydro generating assets, IESO and OPA 
operating costs, and transmission and distribution 
charges. 

Rates for distribution costs are set using a com- 
bination of two mechanisms, as follows: 

• The Cost of Service (COS) review sets rates for 
each distributor every four years or whenever 
the Board deems necessary (the Board has 
also allowed distributors to apply for more 
frequent COS reviews). COS applications are 
detailed and require documentation and cal- 
culations supporting the applicant's electricity 
demand forecasts, estimates of the cost to ser- 
vice this demand, and past operating revenue 



Figure 6: Rate-setting Adjudication Process at the Board 

Source of data: Ontario Energy Board 
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and costs. A typical COS application runs to 
between 800 and 1,200 pages for a small to 
mid-sized local utility. 
• The Incentive Regulation Mechanism (IRM) is 
an annual process that, between COS reviews, 
adjusts rates. It does so by applying a formula 
that considers inflation and productivity. 
Other factors may also be considered in the 
annual rate adjustment on a case-by-case 
basis. A typical IRM application for a small 
to mid-sized utility would require 80 to 100 
pages of documents, including a summary 
with all requested rate adjustments, the mod- 
els used to calculate the new rates, and a list 
of all current rates and charges. 
On average, the Board adjudicates 20 COS 
applications and 60 IRM applications each year for 
Ontario's 80 distribution utilities. 

The rates for transmission (primarily through 
Hydro One) and OPG payments for its regulated 
assets are set using the COS mechanism, and the 
IESO and OPA operating costs are subject to annual 
reviews by the Board. 

As mentioned in our Audit Objective and Scope 
section, the individual Board decisions were not a 
focus of this audit, but we did observe that Board 
staff undertook to provide Board members with 
useful analyses and input to assist them with their 
deliberations. 

Complexity and Cost of Regulatory 
Oversight 

Regardless of their size, all utilities are expected to 
meet the same filing guidelines. We found that this 
"one-size-fits-all" approach to rate-setting is a costly 
exercise that seems to focus as much on getting 
complete records into the public forum as on ensur- 
ing that the process has the information it needs to 
set just and reasonable rates. In addition, all costs 
of the regulatory process must be recovered from 
consumers through rate increases. 

The Board cited customer-service-quality 
statistics for utilities that had gone through COS 



reviews in 2008 or 2009 as evidence that utilities 
can cope with these requirements. However, staff 
of distribution utilities told us that meeting filing 
requirements required significant overtime. In 
addition, small and mid-sized utilities often had to 
engage costly external consultants to help complete 
their applications. Meeting the documentation 
requirements has been particularly challenging for 
the smaller utilities, some of which have fewer than 
2,000 consumers and only five or fewer administra- 
tive staff. We further noted that the Board used to 
provide utilities with rate-application templates 
but no longer does so, providing them instead 
with models, suggested data formats, and filing 
guidelines, which, we were advised, were more 
complicated to use than the templates. 

The average cost of filing a COS rate applica- 
tion is approximately $100,000 for a small utility 
and $250,000 for a mid-sized one, representing 
between 15% and 55% of the revenue increase 
these utilities are seeking in the first place. Most of 
these costs relate to consulting and legal services to 
assist with preparation of evidence to meet Board 
filing requirements, to answer questions from inter- 
veners, and to pay intervenor billings. The cost of a 
rate application for the biggest utilities can run to 
$1 million or more. The impact of this cost ranges 
from about $1 per consumer for the largest utilities 
to as much as $40 per consumer for the smaller 
ones. These amounts are recovered from ratepayers 
over a four-year period. 

The Board had not analyzed the cost/benefit 
impact of its current regulatory requirements in pro- 
tecting consumers. The Board did acknowledge the 
problems faced by smaller utilities in dealing with 
filing requirements but said that every consumer in 
Ontario deserved the same level of protection. 

Intervenors 

Intervenors are individuals or groups of individuals 
who actively participate in the regulatory processes. 
Intervenors may include consumers, consumer and 
trade associations, environmental groups, public 
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interest groups, and affected individuals. The costs 
of their participation in the regulatory process are 
borne by the regulated entities and, eventually, con- 
sumers. Intervenor costs can range from $10,000 for 
a small utility with one intervenor to over $1 million 
for a larger applicant with more intervenors. 

Prior to the start of proceedings, intervenors 
may apply to the Board to have their costs paid by 
the rate applicants. A Board panel rules on a case- 
by-case basis whether intervenors are eligible for an 
award of reasonably incurred costs, which include 
time spent reviewing evidence and participating in 
hearings, and travel and accommodation expenses. 
Because the focus of our audit was not on individ- 
ual Board decisions or judgment, our observations 
relate only to concerns we noted regarding the 
administrative processes — not to individual panel 
decisions or intervenor costs the Board had agreed 
to have applicants reimburse. 

The intervenor community is composed of a 
small number of specialists, primarily lawyers, and 
we recognize that their knowledge and experience 
can add value to the process. However, it is also 
important that intervenors be integrated efficiently 
and effectively into the hearing process to ensure 
that the value they provide is not outweighed by 
the additional costs they impose on consumers, 
who ultimately pay for their services. 

The rate applicants with which we met indicated 
that better co-ordination between Board staff 
and intervenors was needed to manage the heavy 
volume of questions and requests for information 
stemming from intervenors. The applicants also 
noted that there is significant overlap between the 
questions and requests from the intervenors and 
the Board staff; intervenors are recycling questions 
or requests for information from other rate cases 
and, in some instances, the name of the previ- 
ous applicant had not even been removed from 
the questions; and the intervenor questions and 
requests were not always relevant or of significant 
importance to the current case. This last point was 
echoed by the Board in its 2011 OPG decision, 
which raised the concern that an inordinate focus 



on lower-priority issues diminishes the time and 
resources available to pursue the more substantive, 
higher-priority issues. As well, intervenors bill for 
the time that their external consultants and legal 
advisors spend, and all such billings are eventually 
paid by electricity consumers. 

Total intervenor costs over the last three years 
were $16 million for the electricity sector. The rea- 
sonableness of intervenor cost claims can be chal- 
lenged either by the Board or by the rate applicant, 
and there have been 17 claim reductions totalling 
about $750,000 against intervenors over the last 
three years. However, utilities and other applicants 
advised us that they felt this did not reflect the full 
extent of questionable cost claims. They also said 
that they were generally unwilling to challenge 
intervenor billings because they did not want to 
incur the additional costs of such challenges. 



RECOMMENDATION 1 



To enhance the cost-effectiveness of its rate-setting 
process, the Ontario Energy Board should: 

• work with the regulated entities to address 
their concerns about the cost and complexity 
of the current rate-setting filing requirements 
and the impact on their operations; and 

• better co-ordinate and evaluate intervenor 
participation in the rate-setting process in an 
effort to reduce duplication and time spent 
on lower-priority issues. 

| BOARD RESPONSE 

The Board is committed to improving the 
efficiency of its processes, which the Auditor 
General has recognized as being transparent 
and as benefiting from the work of staff and the 
contribution of intervenors. The rate-setting 
process requires appropriate information on the 
public record to support sound and responsible 
decision-making. We annually update our filing 
requirements for rate applications to ensure that 
only appropriate information is being requested. 
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We will continue to consult with the industry 
and other stakeholders to ensure that our rate- 
setting processes are as efficient as possible. 



contracts are outside the scope of its statutory 
mandate and regulatory powers, so any eventual 
approved IPSP would have no impact on procure- 
ment commitments already made by the OPA. 



CHARGES NOT SUBJECT TO 
REGULATORY OVERSIGHT 

Non-regulated Electricity Charges 

In recent years, rates for the electricity component 
of the average bill that is supplied by unregulated 
sources have been significantly higher than rates 
for that supplied by regulated sources, which must 
be approved by the Board. As a result, although 
unregulated electricity accounts for only 50% 
of the total electricity supplied, the price of the 
unregulated electricity accounts for about 65% of 
the price paid by the average consumer. Accord- 
ingly, only about $35 of every $100 in the cost-of- 
electricity component on a typical bill is subject to 
rate regulation by the Board. 

The unregulated sources are primarily suppliers 
under power contracts that have been signed by 
the OPA under the government's direction, because 
the province's long-term power-system plan has not 
been approved by the Board. On August 29, 2007, 
the Board received the OPA's application for review 
and approval of the Integrated Power System Plan 
(IPSP), the blueprint for electricity in Ontario. The 
IPSP must be approved by the Board before the 
plan can be implemented. However, the hearing 
was adjourned on October 2, 2008, pending new 
government targets requiring a revised IPSP, and 
the Board was directed by the Minister of Energy 
on February 17, 2011, to complete its review of the 
OPA's revised IPSP within 12 months of its submis- 
sion. As of August 2011, the revised IPSP had not 
been submitted to the Board for review. 

Over the last four years, the government has 
directed the OPA to enter into new long-term 
electricity-supply contracts in the absence of an 
approved IPSP, which would have set out guidelines 
for such transactions. According to the Board, these 



Non-regulated Regulatory Charges 

There are a number of components in the regula- 
tory charge, including service charges to cover 
the cost of administering the wholesale electricity 
market and maintaining the reliability of the overall 
electricity grid. These charges account for about 
half of the total regulatory charges collected. Other 
components include the operating costs of the 
IESO and OPA; the cost associated with funding 
government conservation and renewable-energy 
programs; a charge to subsidize consumers living 
in rural and/or remote areas; and a charge to help 
recover utility administration costs. 

Most regulatory charges are not subject to any 
form of Board oversight. The exceptions are the 
costs to operate the IESO and OPA, which account 
for about $190 million of the close to $900 million 
in regulatory charges collected annually. The other 
charges either are prescribed by government regu- 
lation or consist of other costs not subject to Board 
oversight. 

Market Surveillance Panel 
As noted earlier, the only regulatory charges in an 
electricity bill whose rates the Board regulates are 
the fees that the IESO and OPA charge to cover their 
operating costs. The Board does not regulate any of 
the other costs of operating the wholesale market. 
The Market Surveillance Panel (Panel), which was 
transferred from the IESO to the Board in 2005, 
monitors wholesale market activities and reports on 
them to the Board twice a year. The Panel has con- 
sistently recommended that the IESO explore struc- 
tural changes to the electricity market to reduce or 
eliminate what are known as "congestion manage- 
ment settlement credit (CMSC) payments" where 
they do not contribute to market efficiency. These 
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payments are a result of the current electricity 
market structure, which compensates generators or 
traders when, for example, transmission constraints 
curtail their ability to participate in the market. 

From 2006 to 2010, the IESO paid more than 
$420 million in constrained-off CMSC payments to 
generators and traders whose power cannot be fed 
into the grid because of the transmission system's 
capacity constraints. In its May-October 2010 
report, the Panel reported that it had two ongoing 
investigations into these market activities. One was 
at the request of a market participant, and the other 
a formal investigation of potential "gaming" of the 
system to obtain increased CMSC payments. 

The Board advised us that, although the Panel 
reports to the Board, it is up to the IESO to imple- 
ment Panel recommendations. However, given 
that the Panel is required to report to the Board, 
we questioned why the Board would not be more 
proactive in ensuring that the IESO gives adequate 
priority to Panel recommendations. In March 2011 
we noted that, for the first time since assuming 
responsibility for monitoring the market in 2005, 
the interim Chair of the Board asked the IESO to 
report back on its proposed response to certain 
Panel recommendations. 

Non-regulated Debt Retirement Charge 

When Ontario Hydro was broken up in 1999, the 
government created the Ontario Electricity Finan- 
cial Corporation (OEFC) to assume its $38.1-billion 
debt and other liabilities and provided it with 
$18.5 billion in financial assets. The difference 
between the assets and debt, $19.4 billion, came 
to be known as the "stranded debt." The govern- 
ment established a long-term plan to repay most 
of it using future electricity revenues, including 
the profits of OPG and Hydro One in excess of the 
government's financing cost for its investment in 
the two entities. 

However, the government also said at the time 
that these anticipated repayment streams would be 
insufficient for an estimated $7.8-billion portion of 



the stranded debt known as the "residual stranded 
debt." In order to repay this amount, the govern- 
ment imposed a new debt retirement charge to be 
included on electricity bills and used to service the 
residual stranded debt. 

The original 1999 plan estimated that the 
stranded debt would likely be retired by 2010. 
However, since then, the OEFC has faced a number 
of challenges in managing the stranded debt, which 
have included the impact of interest charges on the 
$38.1 billion in assumed liabilities, volatility in OPG 
and Hydro One profits, and other government- 
mandated electricity expenditures. As a result, 
OEFC currently estimates that the stranded debt 
will be eliminated between 2015 and 2018. For 
additional information on the stranded debt and 
the debt retirement charge, see Section 3.04, Elec- 
tricity Sector — Stranded Debt. 

The Board has had no role in setting or other- 
wise regulating the debt retirement charge. How- 
ever, given that the Board regulates the industry, 
consumers could reasonably assume that it is 
responsible for overseeing all facets of their electri- 
city bill. To prevent this misconception, the Board 
should clearly spell out charges over which it has no 
power and identify which entities do have control 
over these charges. 



RECOMMENDATION 2 



To help ensure that the interests of consumers 
are protected with respect to those charges not 
subject to Ontario Energy Board (Board) over- 
sight and regulation, the Board should: 

• encourage the Ministry of Energy (Ministry) 
and the Ontario Power Authority (OPA) to 
consult with it on a more timely basis with 
respect to the interests of consumers in all 
energy-supply and pricing undertakings by 
the Ministry and the OPA; 

• work more proactively with the Independent 
Electricity System Operator to address the 
high-priority recommendations from the 
Market Surveillance Panel; and 
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• clearly explain the reason for each charge 
on consumer power bills, identify the entity 
receiving the proceeds from each charge, 
and disclose whether the Board has any 
oversight role relating to the charge. 




The Board supports the objective of enhanced 
co-ordination among energy-sector agencies, 
while at the same time respecting both its own 
mandate and the authority and responsibilities 
of other agencies. The Board will work with 
the Independent Electricity System Operator 
to ensure that high-priority recommendations 
made by the Market Surveillance Panel are 
appropriately addressed in a timely manner. The 
Board has already developed several innovative 
consumer education tools (such as the on-line 
bill calculator) and will examine how to assist 
consumers further. 



CONSUMER PROTECTION 

Consumer Education 

As noted previously, the government enacted a 
regulation in 2004 that required electricity bills 
issued to residential and small-business consumers 
to be broken down by electricity, delivery, regula- 
tory, and debt retirement charges. However, these 
components typically have to be further divided 
into sub-components to be fully explained. 

Given the increased complexity on residential 
electricity bills, consumers need additional sources 
of information to help them understand just what 
they are being asked to pay for. Such education is 
crucial as the sector continues to evolve and con- 
sumers are given more choices in how to manage 
their power costs. For example, they need to under- 
stand the risks and potential benefits of signing 
retail fixed-price contracts. They also need to under- 
stand the time-of-use system and how they may save 
money by adjusting their power-usage patterns. 



Although the Board has indicated that consumer 
education is a responsibility it shares with other 
entities in the electricity sector, the Board has 
established a number of educational programs and 
communication tools, including consumer outreach 
programs, advertising campaigns, and on-line 
resource materials. The Board has also included a 
bill calculator function on its website that enables 
consumers to calculate a monthly estimated bill 
with their local utility or to compare how their 
charges would differ on a retail contract. This is a 
beneficial tool for consumers who want to under- 
stand the price differences between a retail contract 
and the Regulated Price Plan (RPP) before commit- 
ting to a long-term fixed-price contract. A sample 
from the bill calculator is given in Figure 4. 

Although we acknowledge that some of these 
programs have garnered recognition from industry 
associations, there is still room for improvement. 
For instance, in a focus group conducted in 2010, 
many participants said that they still did not under- 
stand the meaning of the charges on their elec- 
tricity bills and were unaware of the Board's role 
in protecting them. In a 2010 stakeholder survey, 
respondents rated the Board poorly on its consumer 
and public education efforts, and similar results 
were noted in focus groups from previous years. 
A continuing lack of understanding of the nature 
of electricity charges by the general public clearly 
poses challenges for the Board in providing assur- 
ance to the public that the interests of electricity 
consumers are being protected. 

We agree that consumer education is a respon- 
sibility that is shared with other entities in the 
electricity sector; however, the Board could use 
its authority over these entities to better influence 
them to meet their responsibilities. 

Monitoring for Compliance 

Regulated entities are required to adhere to the 
accounting, reporting, regulatory, and record- 
keeping requirements specified in the terms and 
conditions of their licences. Regulatory requirements 
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cover a wide range of activities, including conduct 
toward consumers by the regulated entities, bill- 
ing practices and calculations, and related-party 
transactions. 

The Board conducts compliance activities to 
ensure that regulated entities are adhering to their 
statutory and regulatory obligations, and it works 
to ensure that entities understand their obligations. 
It also investigates allegations of non-compliance, 
and undertakes enforcement action where it deems 
appropriate. 

Three Board groups are responsible for com- 
pliance. The Regulatory Audit and Accounting 
Department focuses on ensuring that utilities use 
appropriate accounting policies and practices to 
generate reliable data for regulatory decision- 
making, and conducts audits to ensure that data 
collected from regulated entities is reliable to use in 
decision-making. The Regulatory Policy Group and 
the Consumer Protection Unit assess for compliance 
by monitoring the complaint process and identify- 
ing issues from other sources. They also conduct 
follow-up work, where warranted, on issues they 
have identified. 

Compliance with Reporting Requirements 
The Regulatory Audit and Accounting Depart- 
ment (Department) audits selected accounts and 
service-quality information reported by regulated 
entities. In the last three years, the Department has 
identified consistent deficiencies in utility record- 
keeping and reporting practices and persistent 
difficulties in meeting regulatory accounting and 
reporting requirements. Over the last two years, the 
Department has attempted to address some of these 
weaknesses by organizing three on-line training 
seminars for regulated entities. 

In addition, local utility companies advised us 
that they had concerns about some of the reporting 
requirements. For example, they are not clear why 
some of the requirements even exist, or whether the 
Board uses the information it gets. They also noted 
issues with the required frequency of reporting, 
including a Board requirement that utilities report 



certain information on a quarterly basis, including 
the number of consumers by rate class, the energy 
sales in kilowatt hours for each rate class, and the 
energy sales by electricity retailer. The utilities said 
that there is no need to report this information 
on a quarterly basis, because the industry does 
not change materially within such a short period 
of time. Instead, they said, it would be more cost- 
effective to report on an annual basis. Our review of 
the information collected by the Board also shows 
that the Board did not use this and other reported 
information on a quarterly basis. 

The Board also collects, reviews, and analyzes 
information submitted by utilities to assess the 
reliability and quality of their service and to 
monitor their financial health. However, it has 
not clearly communicated to them why it needs 
the information and how the information is used. 
Such communication would help regulated entities 
understand the reporting requirements and ensure 
that they report correctly, which in turn could also 
enable the Board to identify systemic concerns that 
warrant its attention. 

Compliance with Regulatory Requirements 
In July 2009, the Board's compliance functions 
became the responsibility of its Regulatory Policy 
Group, which has not since conducted any pro- 
active reviews of whether electricity utilities are 
complying with specific regulatory requirements. 
We noted that the current monitoring for compli- 
ance with codes and guidelines relies primarily on 
outside feedback, mostly customer complaints, and 
issues noted in the review of rate applications. 

The last proactive reviews for conditions of 
service and affiliated relations (that is, related- 
party transactions) were conducted in 2007. These 
reviews noted a number of non-compliance issues. 
Among them: 

• Some local utilities unduly transferred 
financial benefits to their affiliates. Examples 
included a $l-million interest-free loan and 
inappropriate sharing of employees between 
the utility and the affiliate. 
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• The Ontario Energy Board Act, 1998 (Act) bars 
distributors from carrying on certain activ- 
ities. Some utilities' provision of municipal 
street lighting was in contravention of the Act. 
Because the Board had not done any recent work 
relating to affiliate transactions, we conducted an 
analysis of affiliated loans currently reported by 
local utilities and selected 10 for follow-up. We 
noted three errors in the information provided to 
the Board regarding these loans, including mistakes 
in reported interest income, loan-related expenses, 
and loan balances. Although the Board agreed that 
these were indeed reporting errors, it also indicated 
that they were identified in the rate-setting applica- 
tions and were therefore taken into consideration 
in the rate-setting process. However, because we 
looked at only one narrow area, it is possible that 
there are errors in other information reported to 
the Board. Without more proactive surveillance, 
such errors could be difficult to detect. 

Consumer Complaints 

The Board's responsibilities include responding 
to inquiries from electricity consumers about the 
Board and dealing with consumer complaints about 
regulated entities. Consumers can contact the 
Board by telephone, on-line, or in person. The num- 
ber of complaints against regulated entities in the 
electricity sector grew from 1,400 in 2006 to 4,300 
in 2010, and totalled 17,000 over the last five years. 
Complaints against electricity retailers account 
for between 70% and 90% of the total, with the 
remainder primarily about local utilities. 

Common complaints include customers being 
switched to retail pricing without a contract, which 
can happen when a retailer obtains a customer's 
electricity account number; misrepresentation of 
identity by retailer agents claiming to work for the 
Board or the local utility; refusal to cancel contracts; 
misrepresentation about retail-contract pricing; and 
even forgery of signatures on the contracts. 

The Board's Consumer Relations Group resolves 
most complaints by contacting the regulated entity 



and by encouraging consumers to try to resolve the 
complaints directly with the company. Complaints 
that cannot be resolved in this way are escalated 
for review and follow-up by the Retail Markets and 
Compliance Management Group. The Board was 
unable to provide data from before 2006, but it said 
that in the last four years, 1,442 cases, representing 
about 11% of complaints against electricity retail- 
ers, were escalated for follow-up. In the last three 
years, 658 electricity retail contracts were cancelled 
through the complaint process and consumers 
received refunds worth more than $700,000. 

Given the continuing high number of complaints 
against electricity retailers, along with the costs 
involved in pursuing enforcement actions, it would 
be helpful for the Board to determine the under- 
lying causes of these complaints and to determine 
whether appropriate mitigation measures can be 
implemented. 

In 2010, the province passed the Energy Con- 
sumer Protection Act to ensure that Ontarians 
have the information they need about electricity 
contracts and electricity pricing, and that they can 
count on fair business practices. The new rules 
came into effect in January 2011, and the Board has 
contracted an external accounting firm to perform 
compliance audits on retailers with respect to the 
new requirements. The related costs of these audits 
(together with most of the costs of operating the 
Board's Consumer Protection Unit) are being allo- 
cated and charged back to retailers and marketers 
through the Board's cost-assessment process. This 
new allocation is effective as of April 1, 2011, in 
accordance with amendments to the Board's cost- 
assessment regulation. 

Retail Contracts 
In the current electricity market, consumers can 
purchase their supply of electricity for consump- 
tion either through their utility at the Regulated 
Price Plan (RPP) rates set by the Board or through 
an electricity retailer at a price set by the retailer. 
There are currently nine active retailers in Ontario, 
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and approximately 630,000 residential consumers 
(representing 15% of the total) have entered into 
contracts with them. 

The Board licenses all retailers who sell electri- 
city contracts in Ontario but does not set the prices 
they charge. The Board indicated that the exist- 
ence of the retail sector and its ability to conduct 
door-to-door sales are matters for the government. 
The Board also indicated that there are inherent 
difficulties in taking enforcement action against 
door-to-door salespeople, given that there is always 
a question of "who said what." However, because 
the Board licenses these entities, we believe that 
the public could reasonably expect it to play a more 
proactive role in protecting consumers from unfair 
business practices. 

Consumer Desire for Price Protection 
Consumers generally enter into retail contracts 
because they want price protection and stability in 
their electricity bills. However, such contracts do 
not actually offer protection against price increases. 
The potential protection they offer is applicable 
only to the "market price" portion of the electricity 
charge on the bill. They provide no protection 
against increases either in the Global Adjustment 
component of the electricity charge or in other 
costs. As noted earlier, the Global Adjustment has 
been rising steadily over the last few years with the 
cost of acquiring the electricity supply, even though 
the overall market price has been declining because 
of oversupply. Most consumers do not follow these 
developments, something that some retailers 
appear to have exploited to encourage consumers 
to sign a contract with them. 

As the government moves forward with its long- 
term energy plan, Ontarians can expect continued 
increases in the cost of electricity. Most of these 
increases will be the result of upgrades to existing 
generating and transmission capacity, and commit- 
ments to purchase renewable energy through long- 
term contracts. As long as there is surplus capacity, 
the price increases associated with many of these 



investments will likely be reflected in the Global 
Adjustment and not the market price. Accordingly, 
consumers with fixed-price contracts will have no 
protection from these increases even though such 
"fixed-price" protection was undoubtedly why con- 
sumers signed these contracts in the first place. In 
fact, the OPA is projecting electricity surpluses in the 
future that will put further downward pressure on 
the market price. Fixed-price contract holders will 
obtain no benefit from any such decreases because 
they will continue to pay their contracted price. 

Effectiveness of Price Protection 
We sampled customer bills from 2006 to 2009 from 
various retailers, and noted that retailers offered 
fixed electricity rates in the range of 8.49(t/kWh to 
10.53(t/kWh. During this same period, the average 
market electricity rate ranged from 3.2(t/kWh to 
5.2(t/kWh. The Board set the average RPP price, 
including both the market and Global Adjustment 
rates, at between 5.4ct/kWh and 6.3(t/kWh. Accord- 
ingly, our sample of retail-contract customers paid 
anywhere from 35% to 65% more for their electri- 
city, before tax and other charges, than the highest 
RPP rate over the term of their contract. 

For example, a consumer who committed to a 
five-year fixed-price electricity retail contract at 
8ct/kWh would have actually seen more dramatic 
electricity price increases and price fluctuations 
on his or her electricity bill than a customer who 
stayed on the Board's Regulated Price Plan, as 
shown in Figure 7. This effectively negates the main 
reason — price stability — that leads people to enter 
into such contracts in the first place. Over the term 
of a five-year contract, we estimate that under this 
scenario a customer using 1,000 kWh per month 
could pay about $2,000 more for electricity than 
one on the RPP plan. As well, retailers have profited 
without facing some of the usual business risks 
because the utilities that supply electricity to the 
retailers' customers are required to pay the retailers 
first and then attempt to collect from consumers. 
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Figure 7: Electricity Price Comparison (RPP vs. Retail-contract Price), 2006-2011 
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As noted earlier, approximately 70% to 90% 
of all customer complaints in the electricity sector 
to the Board over the last five years were against 
retailers. The Board advised us that dealing with 
retailers choosing to conduct door-to-door sales 
is not within its authority; however, because it is 
responsible for licensing retailers, we believe that it 
has at least some responsibility to protect consumers 
from unfair practices by the retailers it licenses. To 
the extent that the Board's responsibility is shared 
with others, such as the Ministry of Consumer 
Services, it would be prudent to ensure that a co- 
ordinated and effective process is in place for resolv- 
ing consumer complaints about these retailers. 

Enforcement 

In its compliance work, the Board has continually 
observed non-compliance with its regulatory and 



reporting requirements by the regulated entities. 
Some of these instances of non-compliance might 
be addressed through better communication, 
such as the on-line training sessions put on by 
the Board's auditing group and the information 
bulletins it puts out. Adequate follow-up reviews 
are also required, to ensure that these and other 
remedial actions have been effective in ensuring 
compliance. 

In addition, since assuming the increased 
responsibilities for regulating the electricity sector 
in 1999, the Board indicated that it made a deliber- 
ate and principled decision in the earlier stages 
of its activities to focus on voluntary compliance, 
recognizing that regulated entities required some 
time to understand and adapt to the legal and regu- 
latory requirements and to correct their practices. 
We acknowledge that time is required for regulated 
entities to adapt to new regulatory requirements 
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and that the Board needed to work with these regu- 
lated entities to ensure that they understand and 
build up their capacity to meet these new require- 
ments. However, a voluntary system is effective only 
if it leads to eventual compliance; if non-compliance 
is persistent, other remedial actions are required. 

The Board clearly recognizes the importance 
of enforcement in effectively regulating the near- 
monopoly that is the electricity sector, because its 
business plans and annual reports acknowledge 
the importance of enforcement as a key part of an 
effective compliance function. That said, despite 
the high number of public complaints against elec- 
tricity retailers, we noted little enforcement action 
against retailers with repeat offences. Since July 
2003, the Board has issued only four enforcement 
orders in 2009 and just one in 2010. In total, three 
retailers were fined about $500,000 and had special 
licence conditions imposed on them. The Board 
indicated that enforcement actions are a costly and 
resource-intensive process. 



RECOMMENDATION 3 



To ensure that consumers are protected and that 
they have the information they need to under- 
stand their electricity bills, the Ontario Energy 
Board should: 

• review its current educational and communi- 
cation programs and make the appropriate 
adjustments to meet consumer information 
needs; 

• consider initiating limited proactive compli- 
ance reviews focusing on high-risk areas; 

• work with utilities to streamline reporting 
requirements, including the timing and fre- 
quency of reporting; and 

• determine whether appropriate deterrent 
actions in those areas that have generated 
frequent legitimate consumer complaints 
can be implemented. 



The Board appreciates the Auditor General's 
recognition of its consumer education materi- 
als, and it commits to enhancing them to meet 
changing consumer needs. 

The Board agrees that proactive compliance 
is an important part of a robust monitoring and 
compliance program. The Board has included 
a commitment to this in each of its business 
plans since 2004 and has undertaken focused 
proactive compliance reviews based on a risk 
assessment that includes reviewing consumer 
complaints. The Board's compliance philosophy 
focuses on bringing industry players into com- 
pliance through a multi-faceted process that 
includes enforcement action where appropriate. 
With the passage of the Energy Consumer Protec- 
tion Act, 2010 (Act), the Board has established 
a Consumer Protection Business Unit that is 
focused on ensuring that industry licensees are 
adhering to consumer protection requirements. 
The Board has conducted detailed compli- 
ance inspections of all active retailers and has 
recently initiated enforcement actions relating 
to allegations of failure by retailers to meet the 
requirements of the Act and related regulatory 
requirements. 

The Board has worked to streamline its 
reporting requirements and will further review 
them in consultation with the industry and 
other stakeholders. In the past two years, the 
Board has taken steps to assist distributors 
by enhancing its electronic filing system to 
facilitate reporting, as well as by providing def- 
initions and guidance that promote a common 
understanding of the reporting requirements. 
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PERFORMANCE MEASURES 

Performance indicators can be defined as measur- 
able outcomes that are within an entity's control 
and clearly linked to its objectives. Since the 
2004/05 fiscal year, the Board has developed and 
published an annual business plan with associated 
performance measures. The business plan identifies 
the Board's strategic objectives and the manage- 
ment initiatives to support them. It also sets out the 
activities that the Board intends to undertake over 
the next three years to achieve its objectives, and 
how it will measure its success. The Board's actual 
performance vis-a-vis these performance measures 
is independently reviewed by an external auditor. 

We concluded that this process was well struc- 
tured and offered the potential to be an excellent 
performance-reporting mechanism. However, to 
take full advantage of this process, the Board's 
performance measures need to be more results- or 
outcome-based, rather than process-oriented or 
output-based. For example, the Board's measures 
looked at whether "Regulated Price Plan prices 
have been adjusted as required" and whether "filing 
guidelines for cost-of-service applications will be 
updated." The challenge with process-oriented or 
output-based measures is that they often provide 
little evidence as to the actual achievement of the 
Board's strategic objectives. We acknowledge that 
in its 2011-2014 Business Plan, the Board recog- 
nized the value of moving toward outcome-based 
performance measures. However, no such measures 
had been developed at the time of our audit. 

One of the Board's performance measures is its 
own internal costs, which have been increasing over 
the last 10 years although they have remained more 
stable over the last three years. In addition to the 
Board's operating expenses, the cost of regulation 
also includes such other expenses as the cost of 
intervenors and costs incurred by applicants seek- 



ing approval for price increases. However, neither 
cost has been included in its cost calculations. 
Because all regulatory costs are ultimately passed 
on to the same electricity consumers that the Board 
is mandated to protect, we believe that these costs 
should also be reflected. 



RECOMMENDATION 4 



To improve the reporting of the effectiveness 
and costs of its regulatory activities, the Ontario 
Energy Board (Board) should develop more 
results-based or outcome-based performance 
measures that are aligned with its strategic 
objectives and mandate, and summarize and 
report all of the costs associated with the 
Board's regulatory processes. 



| BOARD RESPONSE 




In its most recent business plan, the Board 
expressed its commitment to moving to 
outcome-based performance measures. The 
Board is working toward the establishment of 
a robust performance-assessment framework 
that will include the collection and assessment 
of indicators and data relating to the impact of 
its decisions and policy initiatives over time. The 
Board appreciates the Auditor General's conclu- 
sion that its current performance-measurement 
process is well structured and will continue 
to use that process in the interim to confirm 
achievement of its business-plan initiatives. 

The Board will, in addition to reporting on 
its own costs, report on cost awards paid to 
intervenors. The Board will explore whether 
information on utility regulatory costs can be 
readily provided by the utilities at a cost that is 
commensurate with the benefits of enhanced 
reporting. 



Chapter 3 



Ministry of Energy 

Section 

Electricity Sector- 
Renewable Energy 
Initiatives 




The government is responsible for setting the legis- 
lative and policy framework over the production, 
transmission, and sale of electricity in Ontario. The 
three key factors that impact its electricity policy- 
setting role are price, reliability, and sustainability. 

The Ministry of Energy (Ministry) is responsible 
for providing the regulatory framework and imple- 
menting the government's electricity policies, and 
does this in part through its oversight of several 
government entities, including: 

• the Ontario Power Authority (OPA), which 
plans and procures electricity supply to meet 
the province's power needs; 

• the Ontario Energy Board (OEB), which 
regulates Ontario's electricity and natural-gas 
sectors; 

• the Independent Electricity System Operator 
(IESO), which is responsible for the day-to- 
day operation of Ontario's electrical system; 

• Ontario Power Generation (OPG), which gen- 
erates electricity through its nuclear, thermal, 
and hydroelectric stations; and 

• Hydro One, which distributes electricity 
across the province. 

One cornerstone of the current government's 
energy policy is the development of a significantly 



greater role for renewable energy in Ontario's 
electricity-supply mix. Renewable electricity refers 
to those sources of energy generated by natural pro- 
cesses. The four major forms of renewable energy 
are: 

• hydro, generated from the movement of 
water; 

• wind, generated by turbines from air currents; 

• solar, generated by photovoltaic cells that 
capture energy from the sun; and 

• bioenergy, generated by burning organic for- 
estry residues and agricultural wastes. 

The Ontario government has proposed an 
increased reliance on renewable energy sources, 
especially wind, solar, and bioenergy, partly to 
replace coal-fired generating plants by the end of 
2014. The installed capacity from different energy 
sources between 2003 and 2018, as projected in 
the Ministry's Long-Term Energy Plan of November 
2010, is shown in Figure 1. 

In keeping with this priority, the government 
enacted the Green Energy and Green Economy Act 
(Act) in May 2009. The intent of the Act, which 
included new legislation and amendments to 
existing laws, was to attract investment in 
renewable energy, promote a culture of energy 
conservation, create a competitive business 
environment, increase job opportunities, and 
reduce greenhouse gas emissions. 
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Figure 1: Installed Capacity of Electricity Supply from 
Different Energy Sources (MW), 2003-2018 

Source of data: Ministry of Energy 
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1. The expected electricity outputs from wind and solar are much lower than 
their installed capacity (see Figure 10). 

2. Projected. 

Both the Ministry and the OPA have played 
an active role in implementing the government's 
renewable energy policies. The Ministry's respon- 
sibilities have focused on the development of 
programs and policies to advance implementation 
of the Act, while the OPA has played a key role in 
planning and procuring renewable energy by con- 
tracting to buy power from developers of renewable 
energy projects. 



Audit Objective and Scope 



The objective of our audit was to assess whether 
the Ministry of Energy (Ministry) and the Ontario 
Power Authority (OPA) had adequate systems and 
procedures in place to: 

• ensure that renewable energy resources 
are obtained in a cost-effective manner and 
within the context of applicable legislation 
and government policy; and 



• implement a balanced and responsible plan 
with respect to renewable energy that pro- 
vides Ontarians with a clean, reliable, afford- 
able, and sustainable electricity system. 

Senior management at the Ministry and the OPA 
reviewed and agreed to our audit objective and 
associated audit criteria. 

We conducted our audit work at the Ministry 
and the OPA. We also visited the system control 
centre of the Independent Electricity System 
Operator (IESO) to help us better understand the 
operation of Ontario's electricity market. 

In conducting our audit work, we reviewed 
relevant legislation, regulations, policies, and 
procedures; analyzed historical and projected 
electricity-related data collected by the OPA and 
the IESO; reviewed analyses conducted by the 
Ministry and the OPA; interviewed ministry and 
OPA staff; met with representatives from the IESO, 
the Ontario Energy Board, and Hydro One; and 
reviewed relevant literature and best practices 
in other jurisdictions. In addition, we engaged 
independent consultants with expert knowledge of 
Ontario's energy sector on an advisory basis. 

We did not rely on the Ministry's internal audit 
service team to reduce the extent of our audit work 
because it had not recently conducted any audit 
work on renewable energy initiatives. 




Historically in Ontario, electricity generation and 
transmission to residential and commercial users 
was largely the responsibility of Ontario Hydro, a 
Crown corporation, and after 1999, its successor 
companies. The responsibility for ensuring that 
these entities provided consumers with electricity 
that was both sustainable over the long term and 
reasonably priced fell to the Ministry of Energy 
(Ministry) and the Ontario Energy Board, an 
independent regulator. The Green Energy and 
Green Economy Act, 2009 delegated a certain part 
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of the responsibility for dramatically increasing 
the province's renewable energy supply directly to 
the Minister of Energy. Under this legislation, the 
government created a new process to expedite the 
development of renewable energy by providing the 
Minister with the authority to supersede many of 
the government's usual planning and regulatory 
oversight processes. 

As a result, the government has been able to 
further its renewable energy policy agenda without 
the delays that these processes can sometimes 
cause. This agenda has included generating sig- 
nificantly more energy from renewable sources 
to replace coal-sourced energy, given its environ- 
mental and health risks. It has also included creat- 
ing jobs in a new "green" energy sector. 

The government's renewable energy initiatives 
have been successful in rapidly increasing the 
amount of renewable power available over the 
next few years. At the same time, however, wind 
and solar renewable power will add significant 
additional costs to ratepayers' electricity bills. 
Renewable energy sources such as wind and solar 
are also not as reliable and require backup from 
alternative energy-supply methods such as gas-fired 
generation. The government was well aware that its 
renewable energy initiatives meant higher costs but 
felt that this was a more-than-acceptable trade-off 
given the environmental and health benefits, as 
well as the anticipated job-creation benefits. 

Some of our observations relating to the imple- 
mentation of the government's renewable energy 
policy were as follows: 

• Ontario is on track to shut down its more than 
7,500 megawatts (MW) — the capacity as of 
2003 — of coal-fired generation by the end of 
2014. Coal-generated power is being replaced 
by nuclear power from refurbished plants and 
by an increase of about 5,000 MW of gas-fired 
generation, with the remainder resulting 
largely from bringing more renewable energy 
online. More significantly, actions taken by 
the OPA and the Ministry to implement the 
Minister's Directives are projected to increase 



renewable energy, mainly wind and solar 
power, to 10,700 MW by 2018. 

• Because the ministerial directions were quite 
specific about what was to be done, both the 
Ministry and the OPA directed their energies 
to implementing the Minister's requested 
actions as quickly as possible. As a result, 

no comprehensive business-case evaluation 
was done to objectively evaluate the impacts 
of the billion-dollar commitment. Such an 
evaluation would typically include assessing 
the prospective economic and environmental 
effects of such a massive investment in renew- 
able energy on future electricity prices, direct 
and indirect job creation or losses, greenhouse 
gas emissions, and other variables. 

• In May 2009, when the Green Energy and 
Green Economy Act (Act) was passed, the Min- 
istry said the Act would lead to modest incre- 
mental increases in electricity bills of about 
1% annually— the result of adding 1,500 MW 
of renewable energy under a renewable pro- 
curement program called the Feed-in Tariff 
program and implementing conservation 
initiatives. In November 2010, the Ministry 
forecast that a typical residential electricity 
bill would rise about 7.9% annually over the 
next five years, with 56% of the increase 

due to investments in renewable energy that 
would increase the supply to 10,700 MWby 
2018, as well as the associated capital invest- 
ments to connect all the renewable power 
sources to the electricity transmission grid. 

• The OPA was designated as the province's 
energy planner, responsible for submitting 
long-term plans to the Ontario Energy Board 
(OEB) for approval. However, the first long- 
term energy plan put forward by the OPA 
since its creation in December 2004 has not 
been approved by the OEB. Although the OPA 
did spend $10.7 million to develop its first 
energy plan, which it submitted to the OEB 
for review in 2007, the government suspended 
the OEB's review of the plan in 2008. In 2010, 
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the Ministry released its own Long-Term 
Energy Plan to provide the OPA with sufficient 
context on the government's policy priorities 
and targets to guide it in its planning. From 
the public's perspective, this could lead to 
some ambiguity as to which entity is respon- 
sible for electricity planning in Ontario. 
Earlier procurement programs for renewable 
energy included competitive bidding and the 
Renewable Energy Standard Offer Program 
(RESOP), which were both very successful 
and achieved renewable generation targets 
in record time. In particular, RESOP received 
overwhelming responses. It was expected 
to develop 1,000 MW over 10 years, but it 
exceeded this target in a little more than one 
year. Although continuing the successful 
RESOP initiative was one option, the Minister 
directed the OPA to replace RESOP with a new 
Feed-in Tariff (FIT) program that was wider in 
scope, required made-in-Ontario components, 
and provided renewable energy generators 
with significantly more attractive contract 
prices than RESOP. These higher prices added 
about $4.4 billion in costs over the 20-year 
contract terms as compared to what would 
have been incurred had RESOP prices for 
wind and solar power been maintained. The 
Ministry indicated that replacing RESOP 
with FIT successfully expedited its renewable 
energy program and promoted Ontario's 
domestic industry. 

Many other jurisdictions set lower FIT prices 
than Ontario and have mechanisms to limit 
the total costs arising from FIT programs. The 
OPA made a number of recommendations to 
lower Ontario's pricing structure. We were 
advised that the government opted for price 
stability to maintain the investor confidence 
required to attract capital investment to 
Ontario until the planned two-year review 
of the FIT program could be undertaken. 
Examples of proposed changes included the 
following: 



• In March 2009, before the passage of 
the Green Energy and Green Economy Act, 
the OPA proposed a reduction of 9% to 
FIT prices for electricity generated from 
ground-mounted solar projects, in line with 
similar practices in some other jurisdictions. 
This could have reduced the cost of the pro- 
gram by about $2.6 billion over the 20-year 
contract terms. The government did not 
apply this reduction. The Ministry informed 
us that such a predetermined price reduc- 
tion ran counter to the government's goals 
of maintaining policy and price stability for 
the initial two-year period. 

• In February 2010, the OPA recommended 
cutting the FIT price paid for power from 
microFIT ground-mounted solar projects 
after the unexpected popularity of these 
projects at the price of 80.2$ per kilowatt 
hour (kWh), the same price as was being 
paid for rooftop solar projects, became 
apparent. This price would provide these 
ground-mounted solar project developers 
with a 23% to 24% after-tax return on 
equity instead of the 11% intended by the 
OPA. The recommended price cut was not 
implemented until August 2010. In the 
five months from the time the OPA recom- 
mended the price cut in February 2010 to 
the actual announcement in July 2010, the 
OPA received more than 11,000 applica- 
tions from developers. Because the govern- 
ment decided to grandfather the price in 
order to maintain investor confidence, all 
of these applications, if approved, would 
qualify for the higher price rather than the 
reduced one. We estimated that, had the 
revised price been implemented when first 
recommended by the OPA, the cost of the 
program could have been reduced by about 
$950 million over the 20-year contract terms. 

• The Ministry negotiated a contract with a 
consortium of Korean companies to build 
renewable energy projects. The consortium 
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will receive two additional incentives over the 
life of the contract if it meets its job-creation 
targets: a payment of $437 million (reduced 
to $110 million, as announced by the Ministry 
in July 2011 after the completion of our audit 
fieldwork) in addition to the already attractive 
FIT prices; and priority access to Ontario's 
electricity transmission system, whose cap- 
acity to connect renewable energy projects is 
already limited. However, no economic analy- 
sis or business case was done to determine 
whether the agreement with the consortium 
was economically prudent and cost-effective, 
and neither the OEB nor the OPA was 
consulted about the agreement. On Septem- 
ber 29, 2009, the ongoing negotiations with 
the consortium were publicly announced, 
and Cabinet was briefed on the details of the 
negotiations and the prospective agreement 
in October 2009. The formal agreement was 
signed in January 2010. 
Surplus generating capacity is necessary 
to meet periods of peak demand, which, in 
Ontario, occur in the summer. Therefore, to 
ensure system reliability, all jurisdictions will 
have surplus power from time to time. Ontario 
deals with surplus-power situations mainly by 
exporting electricity to other jurisdictions at a 
price that is lower than the cost of generating 
that power. Given that demand growth for 
electricity is expected to remain modest at the 
same time as more renewable energy is being 
added to the system, electricity ratepayers may 
have to pay renewable energy generators under 
the FIT program between $150 million and 
$225 million a year not to generate electricity. 
Ontario's electricity transmission and dis- 
tribution systems already operate at or near 
capacity. A higher-than-anticipated number 
of renewable energy projects under the FIT 
program are awaiting connection to the 
distribution grid. As of April 1, 2011, about 
10,400 MW, representing more than 3,000 



FIT applications, cannot be accommodated 
into the existing power grid. 

• Recent public announcements stated that the 
Green Energy and Green Economy Act, 2009 
was expected to support over 50,000 jobs, 
about 40,000 of which would be related to 
renewable energy. However, about 30,000, or 
75%, of these jobs were expected to be con- 
struction jobs lasting only from one to three 
years. We also noted that studies in other 
jurisdictions have shown that for each job 
created through renewable energy programs, 
about two to four jobs are often lost in other 
sectors of the economy because of higher elec- 
tricity prices. 

• Renewable energy sources such as wind and 
solar provide intermittent energy and require 
backup power from coal- or gas-fired gener- 
ators to maintain a steady, reliable output. 
According to the study used by the Ministry 
and the OPA, 10,000 MW of electricity from 
wind would require an additional 47% of 
non-wind power, typically produced by 
natural-gas-fired generation plants, to ensure 
continuous supply. 

| OVERALL MINISTRY RESPONSE 

The Ministry of Energy (Ministry) welcomes 
the Auditor General's recommendations and 
remains committed to providing quality policy 
advice and implementing the government's 
decisions in a manner that is cost-effective and 
promotes system reliability and sustainability. 

The Green Energy and Green Economy Act, 
2009, enacted by the Ontario Legislature and 
authorizing the creation of a Feed-in Tariff 
(FIT) program, represents a fundamental shift 
in Ontario's electricity policy direction. This 
directional shift is consistent with some 88 juris- 
dictions worldwide that have also implemented 
FIT programs. 

Ontario's FIT program was designed to meet 
three key policy objectives: 
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circumstances. The OPA worked to diligently 
and effectively implement the program within 
short timelines. Consistent with the OPA's own 
internal audit, the Auditor General did not find 
any significant issues with the administration of 
the FIT program. From the outset, a mandatory 
review was built in, at the two-year mark, to 
provide a period of program stability as well as 
to recognize that the program would need to 
evolve as both technology and markets matured 
over time. This review, under way in fall 2011, 
provides an opportunity to consider many of the 



• Reduce our environmental footprint (green- 
house gas emissions) by bringing more 
renewable energy online and supporting the 
phase-out of coal by 2014. 

• Better protect the health of Ontarians by 
eliminating the harmful emissions from 
burning coal. In fact, an Ontario independ- 
ent study in 2005 found that coal-fired 
generation costs $4.4 billion annually when 
health and environmental costs are taken 
into consideration. 

o Create green energy jobs and attract scarce 
investment capital to Ontario amidst a global 
recession. 

The uptake of Ontario's FIT program has 
been successful largely due to the government's 
decision to set attractive FIT prices and instill 
investor confidence by not reducing prices or 
making major policy or program changes prior 
to the mandatory two-year review. 

Planning for a stable supply of electricity is 
a complex exercise requiring compliance with 
North American standards. Prudent planning 
requires providing significantly more generating 
capacity than peak demand. By 2016, energy 
supply and demand are projected to match 
closely as nuclear units are taken offline for 
refurbishment. 

The Ministry will continue to work closely 
with the Ontario Power Authority to balance 
energy supply and demand in the next Inte- 
grated Power System Plan and make adjust- 
ments as necessary to ensure reliability. 



| OVERALL OPA RESPONSE 




The OPA supports the Auditor General's rec- 
ommendations with respect to the ongoing 
development and administration of renewable 
energy programs in the province. The Ontario 
FIT program — the first of its kind in North 
America in scope, comprehensiveness, and 
magnitude — was designed and launched in 
2009 in a particular set of economic and policy 



issues raised in the audit. 

The Auditor General also identifies the 
importance of sector- wide collaboration and co- 
ordination for renewable energy development. 
The OPA works closely with the Ministry of 
Energy, Hydro One, the Independent Electricity 
System Operator, local distribution companies, 
and the Ontario Energy Board on renewable 
energy development — for example, through 
the Renewable Energy Supply Integration 
Team — and will continue to do so. This includes 
finding ways to more effectively communicate 
with the public on the costs of renewable energy 
and other types of electricity generation. Finally, 
the OPA is encouraged that the Auditor General 
recognizes the contribution that renewable 
energy is making to support the reduction of 
greenhouse gases in Ontario's electricity system. 



Detailed Audit Observations 



SIGNIFICANT RENEWABLE ENERGY 
COMING ON-LINE 

Building clean, affordable, reliable, and sustainable 
sources of electricity is a top priority for the Ontario 
government. As part of its goals of protecting the 
environment and the health of Ontarians, the 
government has committed to closing all coal-fired 
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plants by the end of 2014. Ontario is on track to 
meet this commitment. Of the 19 units operated 
at five coal-fired plants across Ontario in 2003, 
the Ministry indicated that eight units had been 
closed since that year and two more were to be shut 
down later in 2011. As a result of these closures, 
the installed capacity of coal-fired generation in 
Ontario has been decreasing. It is anticipated that 
more than 7,500 MW of coal-fired installed capacity 
in 2003 will be replaced by nuclear power from 
refurbished plants and an increase of about 5,000 MW 
of gas-fired generation, with the balance coming 
from new renewable energy sources (see Figure 1) . 

Specifically, with the passage of the Green 
Energy and Green Economy Act, 2009, Ontario has 
made progress in bringing more renewable energy 
on-line. According to the Ministry, the installed 
capacity of cleaner renewable energy such as wind, 
solar, and bioenergy has increased from about 160 
MW in 2003 to about 1,700 MW in 2010, and is 
expected to increase further to 10,700 MW by 2018 
(see Figure 1). 

COST IMPACT OF RENEWABLE ENERGY 
ON CONSUMERS 

Rising electricity costs have in the last few years 
been a concern for Ontarians, who saw their power 
bills rise an average of 26% between 2008 and 
2010, mainly as a result of capital investments, 
refurbishment of generating infrastructure, and 
the imposition of the Harmonized Sale Tax (HST). 
The government responded with a 10% reduction, 
called the Ontario Clean Energy Benefit, on the 
monthly electricity bills of households and small 
businesses that took effect on January 1, 2011, and 
that is to last for five years. 

At the same time, mounting concerns about 
the impact of conventional power generation on 
the environment and public health have led many 
to give serious consideration to environmentally 
friendly renewable energy as an alternative. On the 
other hand, renewable energy sources, particularly 
wind and solar, cost much more than conventional 



energy sources. Accordingly, electricity bills are 
projected to rise even further as more renewable 
energy projects start commercial operations in the 
next few years. The following section deals with 
some of the key factors affecting the cost of electri- 
city in Ontario. 

Hourly Ontario Electricity Price (H0EP) and 
Global Adjustment (GA) 

There are five parts to the typical electricity bill: 
electricity charge, delivery charge, regulatory 
charge, debt retirement charge, and HST. The elec- 
tricity charge accounts for the biggest single portion 
of the bill, and it consists of two key components: 

• The Hourly Ontario Energy Price (HOEP) is 
an hourly market price based on supply and 
demand for electricity as determined by a 
competitive process in which generators bid to 
supply electricity into the market. 

• The Global Adjustment (GA) is the difference 
between the market price (HOEP) and the 
guaranteed prices paid to regulated and con- 
tracted generators. It also accounts for the cost 
of the OPAs conservation programs. Guaran- 
teed prices are paid to generators, including, 
but not limited to, nuclear and hydroelectric 
generators administered by the Ontario Power 
Generation (OPG), non-utility generators 
administered by the Ontario Electricity Finan- 
cial Corporation, and gas-fired and renewable 
energy generators contracted by the OPA. 

The OPA has entered into a number of fixed- 
price contracts, resulting in higher-than-market 
electricity prices. Following passage of the Green 
Energy and Green Economy Act in 2009, the OPA 
was directed to significantly expand renewable 
energy by offering very attractive contract prices 
to developers of renewable energy projects. These 
contracts are expected to lead to significantly 
higher electricity charges through the GA portion of 
the electricity bill. Figure 2 shows that: 

• The sum of the HOEP and the GA, repre- 
senting the biggest part of electricity bills, 
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Figure 2: Electricity Charge, 2006-2014 (<t/kWh) 

Source of data: OPA and IES0 
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increased by 25% between 2006 and 2010, 
and is expected to rise another 43% by 2014 
due to rapid growth in the GA. 
• By 2014, the GA is expected to be 6$ per 
kilowatt hour (kWh) — almost two-thirds of 
the electricity charge — and will be almost two 
times more than that year's projected HOEP. 
Based on our analysis of OPA data, renewable 
energy contracts will contribute significantly to 
increases to the Global Adjustment. As illustrated in 
Figure 3, the total GA is expected to increase tenfold 
province-wide, from about $700 million in 2006 to 
$8.1 billion in 2014, when the last coal-fired plants 
are phased out. Almost one-third of this $8.1 billion 
is attributable to renewable energy contracts. 

Public Awareness of the Cost Impact of 
Renewable Energy 

The OPA indicated that consumers have to be 
advised, through appropriate channels, of the 
expected electricity-price increases arising from 
a large number of contracts to buy green energy 
at fixed rates that are significantly higher than 
market prices. However, a number of consumer 
surveys conducted by the government in spring 
and fall 2010 indicated that although consumers 
generally supported renewable energy, they were 



for the most part unaware of its impact on prices. 
Specifically: 

• An OPA survey showed that only 14% of 
respondents thought renewable energy would 
lead to electricity price increases, while 60% 
disagreed that "green energy sources like wind 
and solar are too expensive and unreliable." 

• Ministry surveys found that only a minority of 
respondents linked recent price increases to 
the cost of renewable energy, although many 
respondents did say that they were prepared to 
pay "modest" increases for renewable electricity. 

• Hydro One surveys found that consumers sup- 
ported spending to connect renewable energy 
to the power grid, but were less inclined to 
support electricity bill increases associated 
with these investments. About half said they 
were willing to pay for such investments, but 
only 27% would agree to an increase in their 
electricity bills of more than 5%. 

In May 2009, when the Green Energy and Green 
Economy Act was passed, the Ministry said it would 
lead to modest incremental increases in electricity 
bills of about 1% annually as a result of adding 
1,500 MW of renewable energy under a renewable 
energy program called Feed-in Tariff (FIT) and 
implementing conservation initiatives. In November 

Figure 3: Total Global Adjustment, 2006-2014 ($ billion) 

Source of data: OPA and IES0 
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2010, the Ministry's Long-Term Energy Plan (LTEP) 
included electricity-price forecasts based on the 
effects of all investments in Ontario's electricity 
system. According to the LTEP, a typical residential 
electricity bill would rise about 7.9% annually over 
the next five years, with 56% of the increase due to 
investment in new, cleaner renewable energy that 
would increase the supply to 10,700 MW by 2018 as 
well as the associated capital investments to connect 
renewable power sources to the transmission grids. 

Because the forecasts in the LTEP were not 
specific to renewable energy, we asked the Ministry 
for a detailed breakdown and analysis showing 
the impact of all renewable energy initiatives on 
various components of residential, industrial, and 
commercial electricity bills. As Figure 4 illustrates, 
the impact of renewable energy on monthly 
electricity charges is expected to increase for all 
sectors between 2010 and 2018, especially the 
large commercial and industrial sectors. However, 
the Ministry did not have a similar breakdown 
for the impact of renewable energy on monthly 
delivery and regulatory charges. We also noted 
that although the LTEP and the related pamphlet 
did inform the public that renewable energy would 
increase their electricity bills, the cost impact of 
renewable energy by sector was not disclosed in 
detail. The Ministry informed us that the forecasts 
in the LTEP were based on all-in total costs, which 



are more important to the public than cost data 
relating to the different sources of energy, such as 
renewable energy. 

In addition to the forecasts in the Ministry's LTEP 
and contained in Figure 4, in April 2010, the OEB 
completed an analysis predicting that a typical house- 
hold's annual electricity bill will increase by about 
$570, or 46%, from about $1,250 in 2009 to more 
than $1,820 by 2014. More than half of this increase 
would be because of renewable energy contracts. 



RECOMMENDATION 1 



To ensure that electricity ratepayers understand 
why their electricity bills are rising at a much 
higher rate than inflation, the Ministry of Energy 
(Ministry) and the Ontario Power Authority 
(OPA) should work together to increase con- 
sumer awareness of the concept of the Global 
Adjustment and make more information avail- 
able on the cost impact of its major components. 



| MINISTRY RESPONSE 




The Ministry agrees that consumer awareness of 
electricity costs, and the factors that affect those 
costs, is vital. 

The Ministry will seek to build on its exten- 
sive public education and awareness actions to 



Figure 4: Monthly Electricity Charge Related to Renewable Energy in Different Sectors 

Source of data: Ministry of Energy 
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1 Economic Sector 


Examples 
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residential 


n/a 


800 


2 


31 


small commercial 


convenience store, small dry cleaner, restaurant, 
small retail store 


12,000 


38 


500 


large commercial 


supermarket, shopping mall, large office building, 
hotel 


130,000 


385 


5,000 


industrial 


paper and pulp, automobile, mining, cement, 
iron and steel manufacturing, chemical products, 
petroleum (i.e., refineries) 


61,200,000 


200,000 


2,400,000 
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date. In 2011, these actions included provid- 
ing the following focused information about 
changes to electricity prices to all of Ontario's 
electricity consumers: 

• the "Electricity Prices Are Changing" pamph- 
let, sent to all Ontario households; and 

• a quarterly electricity bill insert titled 
"Ontario Clean Energy Benefit," detailing 
changes to electricity bills. 

The Ministry will continue to work with 
the Ontario Energy Board, local distribution 
companies, the OPA, and its other partners to 
seek opportunities to further increase public 
awareness about energy prices. The Ministry 
will also explore options for an integrated media 
campaign, which could include web postings 
and fact sheets and other opportunities. 




The OPA agrees with this recommendation. 
Information about the Global Adjustment 
(GA) and the relationship between the OPAs 
contracts and the GA is currently available on 
the OPA website. The OPA has started work 
to simplify this information and co-ordinate 
with other electricity organizations to provide 
comprehensive, consistent information about 
the total cost of electricity. The OPA maintains 
updated cost forecasts and has substantially 
completed an update of the Integrated Power 
System Plan, which will contain a detailed 
cost and bill-impact analysis. As the province's 
electricity planner, the OPA could be the logical 
source of independent and credible information 
on costs. 



DEVELOPMENT OF ENERGY PLAN AND 
RENEWABLE ENERGY POLICY 

The OPA was created in December 2004 by the 
Electricity Restructuring Act. One of its key object- 
ives is to ensure the adequacy and reliability of 



Ontario's electricity supply through planning and 
procurement. Under the legislation, the Ministry 
and the OPA would continue to provide the govern- 
ment with advice on the development of renewable 
energy, but the Minister essentially had the author- 
ity to direct the OPA, which minimized the need 
for an analysis of different policy options and an 
assessment of the cost-effectiveness of alternative 
approaches. 

Integrated Power System Plan (IPSP) 

The OPA has since its inception had the statutory 
responsibility to develop an Integrated Power 
System Plan (IPSP) and procurement processes for 
electricity. The IPSP is to represent Ontario's 20-year 
plan to achieve the province's energy goals. The OPA 
is required to submit the IPSP and the related pro- 
curement processes every three years to the Ontario 
Energy Board (OEB), which then must review the 
proposed IPSP to ensure that it is economically pru- 
dent and cost-effective. However, the OEB has never 
approved the first IPSP put forward by the OPA after 
the OPAs creation in December 2004 because of fre- 
quent changes to government policy and planning 
requirements, as illustrated in Figure 5. 

The OEB's review and approval process of the 
OPAs first IPSP, submitted in August 2007, was 
suspended the following year at the direction of the 
Minister, who asked the OPA to revise the IPSP. The 
suspension of the independent regulator's review 
meant that there would be no independent assess- 
ment to ensure that decisions were made in an 
economically prudent and cost-effective manner. 

In November 2010, the Ministry released a docu- 
ment called the Long-Term Energy Plan (LTEP) that 
specified Ontario's energy goals and supply-mix 
to 2030. The Ministry indicated that the LTEP, 
along with a February 2011 supply-mix directive, 
provided sufficient context to guide the OPA in 
planning and developing a revised IPSP. However, 
OPA staff acknowledged that the existence of two 
plans — the Ministry's and its own — could lead some 
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Figure 5: Key Developments in Ontario's Long-term Energy Planning, 2006-2011 

Source of data: Ministry of Energy and OPA 


Date 


Events 


June 2006 


Minister issues first supply-mix directive, which calls for renewable energy capacity of 15,700 MW by 2025, 
and instructs OPA to develop Integrated Power System Plan (IPSP) and maximize the contribution from 
renewable energy sources. 


Aug. 2007 


OPA submits first IPSP, designed to help achieve goals set in the June 2006 supply-mix directive, to OEB for 
review and approval. 


Sept. 2008 


Minister issues a new supply-mix directive, suspending OEB review and approval process of current IPSP and 
requiring OPA to submit a revised IPSP to OEB within six months. 


Mar. 2009 


OPA does not revise IPSP as per the September 2008 supply-mix directive, saying in a letter to OEB that it 
would wait before issuing revised IPSP due to "significant evolution" in the policy environment. 


May 2009 


Green Energy and Green Economy Act, 2009 is passed to accelerate significant additions of renewable energy 
through creation of a Feed-in Tariff (FIT) program to promote renewable energy, in particular wind and solar 
power. 


Sept. 2009 


Minister issues a directive requiring OPA to develop the FIT program. 


May 2010 


OPA Board of Directors notes that a new IPSP is likely needed due to significant changes that have occurred 
since original IPSP was filed in 2007. 


Nov. 2010 


Ministry releases Long-Term Energy Plan (LTEP), a high-level document highlighting Ontario's energy goals and 
supply-mix to 2030. 


Feb. 2011 


Minister issues a new supply-mix directive, which calls for renewable energy capacity of 19,700 MW by 2018, 
and instructs OPA to develop a new IPSP based on the Ministry's LTEP. 



to conclude that the OPA has only limited authority 
as an energy planner and that the Ministry's LTEP is 
Ontario's "true" plan for the future. 

Renewable Energy Initiatives 

In June 2006, the Minister issued the first supply- 
mix directive to increase the province's renewable 
energy capacity to 15,700 megawatts (MW) by 
2025, representing an increase of about 90% over 
the actual installed capacity of 8,200 MW in 2006. 
In February 2011, the Minister issued a new supply- 
mix directive that further increased the renewable 
energy target to 19,700 MW, but stipulated that it 
be achieved seven years earlier than the date set in 
the 2006 directive. In order to achieve these aggres- 
sive new targets, both the Ministry and the OPA 
expeditiously implemented the actions the Minister 
requested in his ministerial directives. Several 
renewable energy initiatives were introduced, as 
illustrated in Figure 6. 



Although the Ministry consulted with stakehold- 
ers in developing the supply-mix directives, the 
LTEP, and the Green Energy and Green Economy Act, 
billions of dollars were committed to renewable 
energy without fully evaluating the impact, the 
trade-offs, and the alternatives through a compre- 
hensive business-case analysis. Specifically, the 
OPA, the OEB, and the IESO acknowledged that: 

• no independent, objective, expert investiga- 
tion had been done to examine the potential 
effects of renewable-energy policies on prices, 
job creation, and greenhouse gas emissions; 
and 

• no thorough and professional cost/benefit 
analysis had been conducted to identify 
potentially cleaner, more economically 
productive, and cost-effective alternatives to 
renewable energy, such as energy imports and 
increased conservation. 
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Figure 6: Summary of Renewable Energy Initiatives in Ontario 

Source of data: Ministry of Energy and OPA 













Capacity as of April 1, 2011 (MW) 


Launch 


Program/ 


Acquisition 








Non- 


Total 


Date 


Initiative 


Method 


D 


escription 


Committed 1 


committed 2 


Capacity 


OPA-contracted Renewable Energy Sources 


June 2004 
June 2005 
Aug. 2008 


Renewable 
Energy Supply 
(RES 1,11, and III) 


request for 

proposals 

(competitive) 


based on confidential pricing 
proposals from bidders 


1,570 




1,570 


Nov. 2006 


Renewable 
Energy Standard 
Offer Program 
(RESOP) 


standard offer 
(pre-set price) 


initiated by ministerial 
direction to remove obstacles 
for small renewable projects 
by setting fixed contract 
prices and simplifying 
contract rules and processes 


916 


- 


916 


Dec. 2007 


Hydroelectric 
Energy Supply 
Agreement 
(HESA) 


negotiation (non- 
competitive) 


initiated by ministerial 
directions that required OPA 
to enter into hydroelectric 
contracts 


2,062 




2,062 


May 2009 


Hydroelectric 
Contract 
Initiative (HCI) 


Oct. 2009 


Feed-in Tariff 
(FIT) and 
microFIT 


standard offer 
(pre-set price) 


initiated by ministerial 
direction to replace RESOP by 
setting higher contract prices, 
with a focus on creating jobs 
and green economy 


3,675 


10,408 


14,083 


Jan. 2010 


Korean 
consortium 3 


negotiation 

(investment 

arrangement) 


privately negotiated contract 
between the Ministry and the 
Korean consortium 


2,500 




2,500 


Uncontracted Renewable Energy Sources 




uncontracted 
hydroelectric 
facilities 4 


n/a 


managed by private 
developers and/or OPG 


5,938 




5,938 


Total 


16,661 


10,408 


27,069 



1. Includes all projects that were offered contracts or have executed contracts, either under construction or in commercial operation. 

2. Includes all projects that have submitted applications, either under review or waiting for review. Does not include projects that have been rejected or withdrawn. 

3. Considered as committed since the Green Energy Investment Agreement was signed in January 2010. 

4. Estimated by subtracting 2,062 MW (HESA and HCI) from approximately 8,000 MW (total hydroelectric capacity) because no complete listing exists of 
uncontracted hydroelectric facilities. 



Electricity Supply and Demand in Ontario 

According to the OPA, Ontario's electricity genera- 
tion capacity has been much higher than demand 
in recent years. Electricity demand has declined 
since 2005 due to the economic downturn, con- 
servation, and declines in the auto, pulp, and paper 
industries, while supply increased mainly because 



of the addition of renewable energy and gas-fired 
resources. The OPA noted that demand is expected 
to remain flat or decline due to continued con- 
servation efforts and uncertain or slow economic 
recovery, while supply is expected to increase as a 
result of significantly more renewable energy com- 
ing on-line. 
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Our analysis of actual and projected data from 
the IESO and the OPA shows that from 2005 to 
2025, installed and effective capacity will continue 
to exceed both average demand and peak demand. 
The OPA did advise us that Ontario will face sig- 
nificant energy uncertainty beyond 2015 as a result 
of the increasing supply of renewable energy, the 
phasing out of coal by the end of 2014, and the 
refurbishment of nuclear units. Figure 7 shows that 
Ontario will experience a temporary supply reduc- 
tion from 2016 to 2020, when all coal-fired plants 
will be closed and some nuclear units will be taken 
out of service for refurbishment. The expected 
increase in renewable energy sources such as wind 
and solar will not effectively address the temporary 
supply reduction. According to the OPA, renewable 



energy sources are not always available during peak 
demand periods due to their intermittency and low 
effective capacity. 

As illustrated in Figure 7, average demand is 
expected to drop from about 18,000 MW to 16,000 
MW and peak demand from about 26,000 MW to 
24,000 MW. In the same period, installed capacity 
(the maximum amount of electricity that can be 
produced by generators) is expected to rise from 
about 30,000 MW to 43,000 MW, and effective cap- 
acity (the portion of installed capacity that can be 
depended upon to produce electricity) is expected 
to grow from about 27,000 MW to 31,000 MW. 
An OEB analysis completed in April 2010 also 
concluded that, by 2016, electricity supply will 
far exceed demand. Despite these anticipated 



Figure 7: Ontario's Installed and Effective Capacity, and Average and Peak Electricity Demand, 2005-2025 (MW) 

Source of data: OPA and IESO 
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* Projected. Significant uncertainty is expected beyond 2015. 

1. Installed capacity is trie maximum amount of electricity that can be produced by generators. 

2. Effective capacity is the portion of installed capacity that can be depended on to produce electricity. 
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surpluses, renewable energy generators who have 
contracts with the OPA will get paid even though 
Ontario does not need their electricity. 

It is critically important that peak demand (the 
highest demand, generally occurring once a year for 
about one hour in July or August) is met reliably. 
Otherwise, the OPA said, the shortfall between 
available supply and peak demand could lead to 
blackouts. Although Ontario has sufficient genera- 
tion capacity to meet even peak summer demand, 
the OPA indicated that it is required to plan for a 
17% reserve margin in excess of peak demand to 
ensure system safety and reliability and to offset 
unexpected events such as changes in demand and 
equipment failure. The North American Electric 
Reliability Corporation monitors whether this 
requirement is being met. 

We noted that the August 14, 2003, blackout in 
Ontario and the U.S. Northeast — the biggest ever 
in North American history — was not caused by any 
electricity shortfall in Ontario. According to a joint 
Canada-U.S. task force, it was actually triggered 
by an unexpected electricity shutdown in Ohio that 
led to a cascade of shutdowns. 

Figure 7 shows that Ontario's effective capacity 
is expected to grow from about 27,000 MW to 
31,000 MW between 2005 and 2025. However, 
we noted that Ontario rarely needs that much 
effective capacity to meet peak demand throughout 
the year. For example, the last time that demand 
in Ontario reached 27,000 MW was in August 
2006 — and then only for two hours in a single day. 
Since 2007, Ontario has not experienced a single 
day in which demand exceeded 26,000 MW, and it 
experienced only two days of demand greater than 
25,000 MW in 2010. Even on July 21, 2011, one of 
the hottest days on record in the Greater Toronto 
Area and many other Ontario cities, demand was 
about 25,000 MW — well below the all-time high of 
27,000 MW reached in August 2006. 



Roles of the OPA and the 0EB 

Even after the breakup of the former Ontario 
Hydro, Ontario's electricity sector continued to 
have a system of checks and balances in place with 
two expert agencies playing key roles — the OPA 
as energy planner and the OEB as regulator. This 
arrangement was intended to ensure that deci- 
sions are made transparently and objectively; that 
consumers get reliable, affordable, and sustainable 
power; and that any energy plan is economically 
prudent and cost-effective. With the Green Energy 
and Green Economy Act, 2009 (Act) giving the 
Minister the authority to direct certain aspects of 
planning and procurement of electricity supply 
through ministerial "directives" and "directions," 
the frequent exercise of this authority has created 
some ambiguity regarding the original mandates of 
the OPA and the OEB from the planning and over- 
sight perspective. 

The OPA: Planning and Procurement 
The OPA is designated as Ontario's energy planner, 
with the authority to procure electricity supply. 
However, the Minister has the authority to issue 
"directives" (which require Cabinet approval) to 
the OPA regarding the supply mix. The Minister can 
also issue "directions" (which do not require Cab- 
inet approval) on specific electricity-related initia- 
tives, such as renewable energy projects. Since the 
creation of the OPA in December 2004, 22 of the 48 
directives and directions issued to it by the Minister 
were partly or fully related to renewable energy. 

The introduction of the Act has affected the 
OPA's role as Ontario's energy planner. Specifically: 
• Before the Act was passed, the Minister had 
the authority to issue directions without 
Cabinet approval to the OPA to procure 
electricity supply. However, this direction- 
making authority was to expire once the 
OEB approved the OPA's first long-term plan, 
or IPSP, which would have specified the 
procurement processes that the OPA would 
use. In essence, the OPA currently has no 
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independent authority to procure electricity 
supply until the OEB approves its IPSP, except 
pursuant to the authority given to the OPA 
through ministerial directions. However, as 
noted earlier, the first IPSP developed by the 
OPA has never been approved by the OEB. 
• Under the Act, the Minister has the authority 
to issue directions related to renewable energy 
without Cabinet approval, and this direction- 
making authority will not expire after an IPSP 
has been approved. Under this authority, 
the Minister can direct certain aspects of the 
OPA's procurement of renewable energy, 
including price and whether to use competi- 
tive or non-competitive procurement. 
The OPA did acknowledge that, as Ontario's 
energy planner, it requires some level of independ- 
ence to allow it to objectively and proactively 
develop alternative options and ideas instead of 
relying exclusively on ministerial directions. 

The OEB: Regulatory and Oversight 
The OEB is an independent regulatory agency 
mandated to protect the interests of consumers 
with respect to the price, adequacy, reliability, and 
quality of electricity service. It is also responsible 
for promoting economic efficiency and cost- 
effectiveness in the generation, transmission, and 
distribution of electricity. Under the Green Energy 
and Green Economy Act, 2009 (Act), the OEB 
was also given a new objective: the promotion of 
renewable energy, including the timely connection 
of renewable energy projects to transmission and 
distribution systems. 

The ministerial direction-making authority has 
limited the OEB's ability to carry out its regulatory 
and oversight role on behalf of consumers with 
respect to renewable energy. The OEB advised us 
that other than the review of the IPSP, it has no 
oversight responsibility over any procurement of 
renewable energy, which has become an increasingly 
important part of Ontario's electricity-supply mix. 
Because the OEB has not yet approved any IPSP, it 



has had no oversight role with respect to renewable 
energy since the creation of the OPA in 2004. Had 
the OEB's review and approval responsibilities with 
respect to the OPA's first IPSP not been suspended, 
the impact of any ministerial directions would have 
been analyzed as part of the OEB's review of the 
IPSP. Many directions related to the procurement 
and pricing of renewable energy have been issued 
since 2008 in the absence of an approved IPSP, and 
the OEB has had no oversight role whatsoever. A 
report in 2009 by the Environmental Commissioner 
of Ontario raised concerns that the OEB will not 
be able to examine the economic prudence and 
cost-effectiveness of any electricity-related initiatives 
introduced through ministerial directions in the 
absence of an approved IPSP. 

Although the OEB has played an oversight 
role in the connection of renewable energy to the 
grid by evaluating construction, expansion, and 
reinforcement projects of transmission and distribu- 
tion systems, its limited involvement in reviewing 
the procurement and pricing of renewable energy 
has limited the effectiveness of its normal role in 
protecting the interests of consumers with respect to 
prices and overall cost-effectiveness in the electricity 
sector. For example, in December 2007 the Minister 
directed the OPA to enter into contracts for certain 
hydro projects that would have the "potential to 
add a new supply of clean, renewable power at an 
acceptable price to Ontario ratepayers." In January 
2010, the OPA was advised that the estimated cost 
for one of these projects had increased substantially, 
from $1.5 billion to $2.6 billion, and there was no 
guarantee that the cost would not continue to rise. 
Given the estimated $l.l-billion cost increase, the 
OPA expressed concerns about whether the project 
would provide value for ratepayers. In February 
2010, at the OPA's request, a direction was issued 
by the Minister, who acknowledged the cost over- 
run but instructed the OPA to proceed anyway. The 
direction noted that the Minister was satisfied that 
the project remained consistent with government 
priorities. The Ministry informed us that under the 
existing regulatory and legislative framework, the 
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OEB would not have had any oversight role with 
respect to this particular project. 



RECOMMENDATION 2 



To ensure that senior policy decision-makers are 
provided with sound information on which to 
base their decisions on renewable energy policy, 
the Ministry of Energy and the Ontario Power 
Authority should work collaboratively to con- 
duct adequate analyses of the various renewable 
energy implementation alternatives so that 
decision-makers are able to give due considera- 
tion to cost, reliability, and sustainability. 

| MINISTRY RESPONSE 

The Ministry will continue to build on its effect- 
ive collaborative working relationship with the 
OPA to provide decision-makers with the best 
advice, giving due consideration to cost, reliabil- 
ity, and sustainability. In developing the Feed-in 
Tariff (FIT) program, the Ministry worked 
closely with technical experts in the electricity 
sector to harness the best policy and technical 
advice. The expert group met regularly from fall 
2008 to summer 2009 to design the implemen- 
tation of FIT. 

The Ministry will continue to build upon 
its existing policy advisory practices, including 
seeking advice and working in co-operation 
with the OPA, as well as the Independent 
Electricity System Operator, Hydro One, and 
Ontario Power Generation; developing policy 
options and costs; and considering international 
practice, experience, and the perspectives 
brought by non-governmental organizations. 

■ OPA RESPONSE 



The OPA agrees with this recommendation 
and will continue to provide the Ministry with 
expert professional advice on the development 
of renewable energy as well as other types of 
generation. The OPA has substantially com- 



pleted an update of the Integrated Power Sys- 
tem Plan (IPSP) and plans to file the document 
with the Ontario Energy Board in fall 2011. 
Cost, reliability, and sustainability of renewable 
energy and other sources of generation are 
assessed in the updated IPSP. 



PROCUREMENT OF RENEWABLE ENERGY 

Procurement Methods 

There have been three forms of procurement pro- 
cesses for renewable energy: competitive (request 
for proposals), non-competitive (negotiations), and 
standard offer (pre-set price), as illustrated in Fig- 
ure 6. Initially, Ontario solicited renewable energy 
projects mainly through competitive requests for 
proposals from private developers. In recent years, 
renewable energy has often been procured through 
standard-offer and non-competitive processes in 
response to ministerial directions. Prices for renew- 
able energy, especially under the FIT program, have 
been between two and 10 times higher than those 
of conventional energy sources, such as nuclear, 
natural gas, and coal. Generators of renewable 
energy will be paid guaranteed prices over the 
contract terms, which range from 20 years for elec- 
tricity from wind, solar, and bioenergy, to 40 years 
for hydroelectricity. 

Request for Proposals and Standard-Offer 
Program 

The first competitive procurement initiative 
adopted by the government to acquire renewable 
energy was several requests for proposals (RFPs) 
inviting potential developers to bid on renewable 
energy projects. The OPA indicated that the com- 
petitive process usually provides the best value 
and is the preferred option, barring other policy 
priorities, to ensure that contracted prices are 
cost-effective and reflect current market costs. 
Three RFPs for Renewable Energy Supply (RES) 
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Figure 8: Prices of Renewable Energy Sources under Different Procurement Methods, as of April 2011 (<t/kWh) 

Source of data: Ministry of Energy and OPA 
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1. Weighted averages of all projects. 

2. Prices vary depending on project size, with smaller projects typically qualifying for higher prices. 

3. Standard FIT prices apply to phase 1 and phase 2 projects, plus additional payment called Economic Development Adder (EDA) as stated in the original 
Green Energy Investment Agreement (GEIA). Subsequent to our audit fieldwork, the GEIA was amended in July 2011, and the EDA was reduced to 1.430/ 
kWh for solar power and 0.27t/kWh for wind power. 



programs were issued: RES I in June 2004, RES II in 
June 2005, and RES III in August 2008. 

However, the complexity and cost of developing 
competitive RFPs was seen as favouring larger 
projects at the expense of smaller ones. To remove 
these barriers to small projects, the Minister issued 
a direction in 2006 to the OPA to develop a Renew- 
able Energy Standard Offer Program (RESOP) that 
would offer smaller renewable energy projects 
a standard pricing regime while providing for 
simplified regulations, including eligibility and 
contracting. 

Prices under RESOP were about 16% to 40% 
higher than the competitive prices under the RFPs, 
as illustrated in Figure 8. The OPA indicated that 
RESOP would not be successful if the standard 
prices were not set high enough to attract invest- 
ment in renewable energy projects. On the other 
hand, the OPA did acknowledge that the standard- 
offer process might have had some unintended 
consequences arising from an absence of the 
competitive tension that encourages innovative 
solutions, and it did ultimately result in high prices 
and oversubscription. 



The Ministry and the OPA indicated that both 
RES and RESOP were successful. For example, 
RES I substantially increased the number of wind 
turbines, from 10 in 2003 to more than 200 in 
2006, an increase in capacity of about 300 MW. 
RES II, which had been intended to attract 1,000 MW 
of renewable energy, had twice as many applica- 
tions as expected because of developers' interest in 
the guaranteed high prices. 

Feed-in Tariff (FIT) Program 
Both RES and RESOP proved to be immediate suc- 
cesses, with high response rates and generation tar- 
gets being met in record time. In particular, RESOP, 
which offered very attractive contract prices to 
renewable energy generators, received overwhelm- 
ing responses. When RESOP was launched in Nov- 
ember 2006, it was expected to develop 1,000 MW 
over 10 years. In May 2008, the OPA indicated that 
RESOP had exceeded all expectations and achieved 
more than 1,000 MW of contracted projects in a 
little more than a year. Although continuing the 
successful RESOP initiative was one option, the 
Minister directed the OPA in September 2009 to 
replace RESOP with a new standard-offer program 
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called Feed-in Tariff (FIT), which was wider in 
scope, required made-in-Ontario components, and 
provided renewable energy generators with signifi- 
cantly more attractive contract prices than RESOP, 
as illustrated in Figure 8. These higher prices 
added about $4.4 billion in costs over the 20-year 
contract terms as compared to what would have 
been incurred had RESOP prices for wind and solar 
power been maintained. The Ministry indicated 
that replacing RESOP with FIT successfully exped- 
ited its renewable energy program and promoted 
Ontario's domestic industry. 

According to the Ministry, RES and RESOP were 
replaced with FIT following a government policy 
decision to expand more rapidly the procurement 
of renewable energy in order to create jobs and 
protect the environment. 

Determination of FIT Prices 

The FIT program aims to encourage development 
of renewable energy projects by a diverse range of 
developers, including homeowners, farmers, small 
businesses, and community groups, by offering 
long-term, fixed prices for the electricity they gen- 
erate. Launched in October 2009, FIT garnered an 
overwhelming response, receiving applications for 
a total capacity of about 14,000 MW at the end of 
the first quarter of 2011. The FIT program has two 
streams: the comprehensive FIT stream for projects 
over 10 kW and the simplified microFIT stream 
for those under 10 kW. Both offer prices that vary 
depending on energy sources (wind, solar, hydro, 
and bioenergy), project sizes (microFIT projects 
below 10 kW qualify for higher prices), and deploy- 
ment methods (rooftop or ground-mounted solar, 
onshore or offshore wind), as illustrated in Figure 8. 

FIT prices were based on several factors, 
including prior experience in Ontario and other 
jurisdictions, feedback from stakeholders, and cost 
assumptions for capital, operations and mainten- 
ance, connection, term of contract, generating 
capacity, and construction lead time. Ontario's FIT 
prices were originally designed with the intention 
of allowing a reasonable rate of return, defined as 



11% after-tax return on equity, for developers of all 
types of renewable energy projects. However, we 
noted that: 

• There was minimal documentation to support 
how FIT prices were calculated to achieve 
the targeted return on equity, because of the 
numerous changes to the financial model and 
assumptions used by the OPA. 

• There has been a lack of independent over- 
sight on the reasonableness of FIT prices. 
Although the OEB has historically been 
mandated to oversee and approve electricity 
prices, it has no role or legislative responsibil- 
ity to review or approve FIT prices. The OPA 
informed us that the first review of FIT prices 
will be conducted in-house by OPA staff, sup- 
ported by consultants as needed, during fall 
2011. However, the Ministry indicated that 
the government has not decided whether 

to involve an independent third party in the 
review. 

The OPA said it initially developed Ontario's FIT 
prices based on the long-established and successful 
FIT programs in Germany and Spain. We noted that 
the internal rates of return offered to the develop- 
ers in these countries varied depending on project 
risks and ranged from just 5% to 7% in Germany 
to between 7% and 10% in Spain. When Ontario's 
FIT prices were first developed in spring 2009, they 
were already higher than those of Germany and 
Spain, which have both significantly dropped their 
FIT prices since then due to lower component costs 
arising from technological advances. However, 
Ontario's prices have remained unchanged, except 
for a drop in the rate for small ground-mounted 
solar projects. According to the Ministry and the 
OPA, it was a deliberate decision by the government 
to maintain price stability in order to retain investor 
confidence and offer very attractive prices to invest- 
ors in order to encourage the start-up of a "green" 
industry in Ontario. 
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Revision of FIT Prices 

By July 2010, less than a year after the launch of 
FIT, the OPA had received more than 16,000 appli- 
cations, about 13,500 of which were for ground- 
mounted solar projects. According to the OPA, this 
overwhelming response highlighted the unexpected 
popularity of microFIT ground-mounted solar 
projects at the price of 80.2(t/kWh, the same price 
that was being paid for rooftop solar projects. The 
original FIT price of 80.2(t/kWh would provide 
developers of these ground-mounted solar projects 
with a 23% to 24% after-tax return on equity 
instead of the 11% intended by the OPA. Therefore, 
in July 2010 OPA proposed cutting the price by 
about 27%, from 80.2(t/kWh to 58.8d/kWh. 

The proposed price cut brought a strong 
response during a 30-day round of consultations. 
Many developers objected to the proposed 58.8$/ 
kWh price and demanded that the OPA grandfather 
the 80.2(t/kWh price for those applications already 
filed. In August 2010, the OPA issued a more 
modest price cut of about 20% — to 64.2(t/kWh 
instead of 58.8(t/kWh — and agreed to pay 80. 2$/ 
kWh for all applications received by the OPA up to 
then, including those still awaiting approval. The 
OPA applied the price cut only to new applications 
in order to ensure price and policy stability and 
prevent any potential lawsuits. We also noted that 
the price cut had limited impact because it was not 
done in a timely way. Specifically: 

• The OPA had proposed since February 2010 
that immediate action be taken to reduce the 
FIT price for ground-mounted solar projects. 
The OPA informed us that the price cut was 
not announced until July 2010, five months 
later, because the government needed time to 
analyze the situation. Due to this delay, the 
OPA received more than 11,000 applications 
from February to June 2010, all of which quali- 
fied for the full price rather than the reduced 
one because of the decision to grandfather the 
price in order to maintain investor confidence. 

• The number of applications for ground- 
mounted solar generation dropped signifi- 



cantly, from more than 2,000 in June 2010 to 
fewer than 200 in August 2010, and remained 
stable at that level thereafter. Because the 
OPA grandfathered the original price of 
80.2(t/kWh for all applications already filed, 
the reduced price of 64.2(t/kWh applied 
only to new applications received after the 
announcement of the price cut in August 2010 
(about 200 per month) . 
In addition, we noted that the revised price of 
58.8(t/kWh originally proposed by the OPA would 
have provided developers with an 11% after-tax 
return on equity intended for all renewable energy 
projects. However, the revised price went from 
58.8(t/kWh to 64.2(t/kWh without adequate 
documentation to support how the OPA arrived at 
the higher price. The OPA indicated that 64.2$/ 
kWh was a reasonable price based on justifications 
provided by developers and other stakeholders. 
We estimated that, had the OPA been successful 
in making the price cut to 58.8(t/kWh when it was 
initially recommended, electricity ratepayers would 
have saved about $950 million over the 20-year 
contract terms, while developers would still have 
received their 11% after-tax return. 

Cross-jurisdictional Comparison of FIT Prices 

Our research found that Ontario's FIT prices were 
generally higher than those of other jurisdictions, 
especially for solar projects, as illustrated in Figure 9. 
According to the Ministry, Ontario's prices were set 
higher than elsewhere to create investor confidence 
and more quickly attract investment capital amidst 
a global recession. A unique feature of Ontario's FIT 
program, the domestic content requirement, also 
led to higher prices because the cost of Ontario- 
made generation components is higher than that of 
comparable equipment made in lower-cost jurisdic- 
tions such as China. 

Our research also noted that many jurisdictions 
have mechanisms in place to control the increase 
of FIT prices. For example, Germany reduces prices 
automatically by a certain percentage every year for 
new projects, while Spain regularly revises its prices 
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Figure 9: Comparison of FIT Prices as of April 2011 (<t/kWh in Canadian $) 1 

Prepared by the Office of the Auditor General of Ontario 
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1. Prices vary depending on project size, with smaller projects typically qualifying for higher prices. Prices were converted to Canadian currency based on the 
average exchange rates in April 2011. 

2. These base rates are increased if the components are manufactured in Washington. 



based on pre-set capacity targets. Washington State 
has imposed an annual maximum payment per 
contractor, while several American and Australian 
states set caps on capacity that, when reached, 
result in termination of a FIT program. 

In Ontario, the government chose to maintain 
price stability until the two-year program review 
could be undertaken rather than incorporating any 
price or capacity adjustment mechanisms such as 
the following: 

• The initial FIT prices proposed by the OPA 
in March 2009, prior to the passage of the 
Green Energy and Green Economy Act, included 
an automatic 9% drop in the contract price 
for every 100 MW of power contracted from 



ground-mounted solar projects. However, the 
OPA informed us that the Minister removed 
this adjustment, fearing that it would discour- 
age manufacturing investments and hamper 
the development of renewable energy. We 
estimated that if this adjustment had been 
implemented as first proposed, the cost of the 
FIT program could have been reduced by about 
$2.6 billion over the 20-year contract terms. 
• The absence of caps or limits to the number of 
contracts signed under Ontario's FIT program 
led to the current oversubscription. The OPA 
informed us that it designed the FIT program 
at a time when no long-term energy plan was 
in place and it was unsure about the quantities 
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of power the FIT program was intended to pro- 
cure. The OEB indicated that ceilings, caps, or 
other measures must be in place to minimize 
the risk of higher consumer prices and less- 
than-optimal deployment of resources. 
Both the Ministry and the OPA were aware 
of the high FIT prices in Ontario and of the price 
reduction and program-control mechanisms in 
other jurisdictions. However, the Ministry indicated 
that the government's decision was not to change 
prices before the first planned review of the FIT 
program — targeted to take place in fall 2011, two 
years after the program's introduction — so as to 
create stability and instill investor confidence. 

However, we noted that in October 2010, the 
OPA did recommend that instead of reviewing the 
FIT program in fall 2011 and making incremental 
changes as issues arise, an "immediate program 
review" should be conducted to ensure that priority 
issues are addressed more fully and that ad hoc 
changes are avoided to preserve the credibility and 
stability of the FIT program. One of the top-priority 
issues identified by the OPA was the significant 
reduction in the cost of solar technologies — about 
50% since 2009 — as the technology matured and 
improved. The OPA specifically recommended 
reducing FIT prices for solar projects to reflect cur- 
rent market conditions and introducing a plan to 
signal further price reductions in future. However, 
the OPA informed us that no decision had been 
forthcoming regarding its concern about the very 
generous prices being offered to investors in renew- 
able energy projects. 

FIT Contract Term: Additional Contract Payment 

A situation called curtailment occurs when the 
Independent Electricity System Operator (IESO) 
instructs generators to reduce all or part of their 
output in order to mitigate an oversupply of 
electricity. Compared to other renewable energy 
contracts such as RES and RESOP, the FIT contract 
has a unique feature that offers renewable energy 
generators an "Additional Contract Payment" to 
compensate them for any revenue lost as a result 



of curtailment instruction. Accordingly, electricity 
ratepayers still have to pay renewable energy 
developers even when those generators are not pro- 
ducing electricity during periods of curtailment. 

The IESO has not yet curtailed renewable energy 
generators under the FIT program because no 
FIT projects have been on-line, and therefore no 
"Additional Contract Payment" has been triggered 
or included in electricity bills to date. However, 
the OPA and the IESO acknowledged that when 
more renewable energy projects under the FIT 
program are added to the grid, the power surplus 
will grow and such curtailments will be likely (see 
"Operational Challenge: Surplus Power" later in 
this report) . 

There has been inadequate assessment of the 
potential costs of curtailing renewable energy, even 
though there is a strong likelihood of curtailment 
in the future for these energy sources. For example, 
the OPA has performed several scenario analyses, 
but none included the impact of curtailing renew- 
able energy. The OPA indicated that its plans are 
based on situations where supply equals demand, 
but not where there are surpluses and where the 
curtailment of renewable energy may be required. 

The OPA also noted that the calculation of cur- 
tailment costs depends on a number of factors and 
assumptions that could be very volatile. The only 
analysis on curtailment we found was done by the 
IESO in 2009. It estimated that the substantial addi- 
tion of renewable energy would result in curtailment 
of between 2,000 and 2,500 hours per year and 
that the cost of paying renewable generators for not 
producing electricity could range from $150 million 
to $225 million a year. However, these projections 
were based on 2008 data and we were advised that 
no updated projections had been done since then. 

Agreement with the Korean Consortium 

While the FIT program was intended to provide a 
channel for renewable energy investments by home- 
owners, farmers, small businesses, and community 
groups, the Ministry was also negotiating with a 
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consortium of Korean companies under separate 
terms to build more renewable energy projects. 

The consortium, led by two large Korean com- 
panies, approached the Ministry in June 2008 and 
proposed to make a major investment in Ontario's 
renewable energy sector. This led to ongoing talks 
between the Ministry and the consortium and 
the signing of a memorandum of understanding 
in December 2008. In June 2009, the Minister 
travelled to Korea for more discussions; six months 
later, the Minister, on behalf of the government, 
signed the $7-billion Green Energy Investment 
Agreement (GEIA) with the consortium. The 
consortium committed to build 2,000 MW of wind 
projects and 500 MW of solar projects in Ontario 
in five phases by 2016, with the equipment to be 
manufactured in this province. 

Neither the OEB nor the OPA was consulted 
about the agreement. The OPA was not involved 
until summer 2009, when the Ministry inquired 
about available transmission capacity to accommo- 
date consortium projects. On September 29, 2009, 
the ongoing negotiations with the consortium were 
publicly announced, and Cabinet was briefed on 
the negotiations and prospective agreement shortly 
thereafter. We were advised that Cabinet had sub- 
sequent briefings prior to finalization of the agree- 
ment in January 2010. In April 2010, the Ministry 
directed the OPA to negotiate with the consortium 
on the Power Purchase Agreements (PPAs), which 
outline contractual obligations and payment terms 
for each renewable energy project to be developed 
by the consortium. As of April 2011, details of 
the PPAs had not yet been finalized. Subsequent 
to our audit fieldwork, six PPAs were signed in 
August 2011. 

The draft PPAs with the consortium are substan- 
tially similar to FIT contracts, but the consortium 
will receive two additional incentives: priority 
access to Ontario's transmission system; and, 
originally, an additional $437 million on top of the 
standard FIT prices, contingent on the fulfillment 
of the consortium commitment to build four manu- 
facturing plants in Ontario. Subsequent to our audit 



fieldwork, the Ministry renegotiated the GEIA with 
the consortium, which had requested a one-year 
commercial operation date extension for phases 
one and two of its projects because of challenges in 
completing its regulatory and environmental stud- 
ies. In July 2011, as a result of the date extension 
and other changes, the Ministry amended the GEIA 
to reduce the additional $437 million payment to 
$110 million. 

According to the Ministry, the consortium agree- 
ment is neither a non-competitive procurement 
nor a sole-source deal. Instead, it is an "investment 
arrangement" with an objective of establishing a 
sound green energy sector in Ontario since no other 
company has proposed to invest in Ontario's renew- 
able energy sector at the size and scale of the con- 
sortium and its partners. However, we noted that 
the normal due diligence process for an expendi- 
ture of this magnitude had not been followed. For 
large projects such as the consortium agreement, 
we expected but did not find that a comprehensive 
and detailed economic analysis or business case 
had been prepared. According to the Ministry, 
the decision to enter into the agreement with the 
consortium was made by the government. Although 
the Cabinet was briefed about the agreement, the 
Ministry indicated that there had been no formal 
Cabinet approval because it was not required. 



RECOMMENDATION 3 



To ensure that the price of renewable energy 
achieves the government's dual goals of cost- 
effectiveness and encouraging a green industry, 
the Ministry of Energy and the Ontario Power 
Authority should: 

• work collaboratively to give adequate and 
timely consideration to the experiences of 
other jurisdictions and lessons learned from 
previous procurements in Ontario when 
setting and adjusting the renewable contract 
prices; 

• work with the Independent Electricity 
System Operator to assess the impact of 
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curtailing renewables as part of its energy 
planning in order to identify ways to optimize 
the electricity market; and 
• ensure that adequate due diligence is 

undertaken, commensurate with the size of 
electricity-sector investments. 

| MINISTRY RESPONSE 

The Ministry will continue to take into con- 
sideration the experiences of other jurisdictions 
while ensuring that the program remains stable 
and sustainable. As planned, the Ministry will 
undertake a mandatory two-year review of the 
Feed-in Tariff (FIT) program (as required in the 
Minister's FIT direction) in conjunction with 
the OPA. The review will examine potential 
FIT price reductions, as well as FIT support 
programs, contract rules, and how the program 
is meeting the government's policy objectives. 
Recommendations for improving the FIT pro- 
gram will be made to the Minister. 

The Ministry will continue to work with the 
Independent Electricity System Operator (IESO) 
during the development of new rules and tools 
to better integrate renewable energy sources 
into the market. This ongoing work includes 
more precise forecasting of load and intermit- 
tent generation and the ability to dispatch (turn 
down or off) renewable energy facilities such 
as wind that until now have been able to run 
whenever they were available to. 

In order to fulfill the Ministry's key objectives 
of electricity reliability, sustainability, and cost- 
effectiveness, the Ministry agrees to continue 
to provide a full analysis of new investments, 
including through the Integrated Power System 
Plan, which is to be updated every three years. 
This will ensure that system planning continues 
to reflect the most up-to-date and accurate 
information and challenges affecting the sys- 
tem. The Ministry will continue to work collab- 
oratively with the IESO, OPA, and all partners 



in the sector to ensure the system is capable of 
meeting new challenges. 

| OPA RESPONSE 

A mandatory two-year review of the FIT pro- 
gram will be carried out in the near future. 
Experience from other jurisdictions and previ- 
ous Ontario procurements will be considered as 
part of the review. 

A reliable and sustainable electricity system 
will from time to time have surplus power. A key 
objective of the OPA, the Ministry, and the IESO 
is to strike the right balance between ensuring 
that clean, reliable electricity facilities are built 
and are available when required, and ensuring 
that ratepayer value is maximized. For the last 
two years, the OPA has been working with the 
IESO and other stakeholders on the issue of 
potential surplus energy and curtailment for 
renewable energy and other types of generation. 
This process has included looking at the appro- 
priate contractual options available to curtail 
resources when necessary at the lowest possible 
cost to ratepayers. The FIT contracts do contain 
curtailment provisions. The OPA and IESO have 
been actively collaborating on aligning other 
renewable energy contracts to make operators 
more responsive to market rules. 

The OPA will continue to perform due dili- 
gence with respect to the design of plans and 
the execution of contracts on behalf of electricity 
ratepayers, and will continue to provide the Min- 
istry and other sector stakeholders with updated 
plans and status and outlook reports. 



Co-ordination and Planningforthe 
Procurement of Renewable Energy 

The development of renewable energy initiatives 
involves planning and co-ordination with other 
parties, including the Ministry of the Environment, 
the Ministry of Natural Resources, federal agencies, 
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and municipalities. We noted several instances 
where renewable energy initiatives led to poten- 
tially unnecessary compensation and potential 
lawsuits because of conflicts with environmental 
impact and planning decisions. Among them: 

• In June 2009, the Ministry of the Environ- 
ment changed the regulations governing the 
placement of wind turbines, affecting some 
onshore wind contracts already awarded by 
the OPA. One developer filed a claim against 
the OPA and, in order to avoid litigation, the 
OPA agreed to settle by paying the developer 
up to $2.4 million. 

• In June 2010, the Ministry of the Environment 
proposed a policy relating to offshore wind 
turbines. In February 2011, the government 
decided to suspend all offshore wind projects 
pending completion of independent scientific 
research. Although this decision affected all 
offshore wind projects under FIT, the OPA was 
not informed of the decision until three days 
before the public announcement. Affected 
developers felt that they had been incurring 
costs in good faith even though the govern- 
ment was planning to suspend offshore pro- 
jects, resulting in ongoing negotiations since 
then between the developers and the OPA. 

• In October 2010, the Ministry cancelled 
a signed contract with a private-sector 
developer to build a 900 MW gas-fired project 
in the GTA because decreased electricity 
demand, the supply of more than 8,000 MW 
of new and cleaner power, and increased 
conservation efforts had made it unneces- 
sary. The OPA has been negotiating with 

the developer to reach agreement over the 
amount of possible compensation to be paid 
for the cancellation of the signed contract. 



RECOMMENDATION 4 



To avoid unintended costs arising out of changes 
to regulatory requirements and changes to sup- 
ply and demand situations, the Ontario Power 



Authority and the Ministry of Energy should 
work collaboratively with other ministries and 
agencies to ensure that they are made aware on 
a timely basis of anticipated policy and regula- 
tory changes. 



| MINISTRY RESPONS 




The Ministry agrees that close collaboration with 
other ministries and agencies on proposed policy 
and regulatory changes is vitally important. 

The government carefully considered, sup- 
ported by scientific research, its policy decision 
to create uniform provincial standards for place- 
ment of wind turbines away from homes. The 
government considered this policy choice to be 
better than having each municipality decide the 
setback distances in an ad hoc way. 

With respect to the offshore wind develop- 
ment, the Ontario government and the U.S. 
Department of Energy have worked collabora- 
tively on developing wind resources in the Great 
Lakes. The collaboration involves joint scientific 
research to inform the creation of a uniform 
regulatory framework and policies. It is neces- 
sary to suspend further offshore projects until 
the scientific research is completed. 

The Ministry will continue to build on its 
existing practice of ensuring strong and regular 
staff connections between relevant ministries, 
recognizing that it can inform agencies or other 
parties of new policy direction only after a duly 
authorized decision is made. 

| OPA RESPONSE 

The OPA agrees with this recommendation and 
continues to work closely with Hydro One and 
the Independent Electricity System Operator to 
assess and manage the impacts of new genera- 
tion on the electricity system. 
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RELIABILITY OF RENEWABLE ENERGY 

Solar and wind energy are by their nature inter- 
mittent, and the growing contribution of these 
unpredictable resources to the energy-supply mix 
has increased uncertainty and created challenges 
for the Independent Electricity System Operator 
(IESO). It has to balance supply and demand to 
ensure that renewable energy can be efficiently 
integrated into the operation of Ontario's power 
system without compromising the reliability, stabil- 
ity, and efficiency of the system. 

The power-generating capacity of a power plant 
can be measured in two ways: "capacity factor" (the 
ratio of the actual output of a power plant in a given 
period to the theoretical maximum output of the 
plant operating at full capacity) and "capacity con- 
tribution" (the amount of capacity available to gen- 
erate power at a time of peak electricity demand, 
which is usually in July and August). 

The power-generating capacity of current wind 
and solar technology is much lower than other 
energy sources, as illustrated in Figure 10. Wind 
generators operate at 28% capacity factor but 
have only 11% availability at peak demand due to 
lower wind output in the summer. Solar generators 
operate at just 13% to 14% capacity factor on aver- 
age for the year but have 40% availability at peak 
demand in the summer. 

We analyzed the performance of all wind farms 
in Ontario in 2010 based on IESO data. Although 
the average capacity factor of wind throughout the 
year was 28%, it fluctuated seasonally, from 17% in 
the summer to 32% in the winter. It also fluctuated 
daily, from 0% on summer days, when electricity 
demand was high, to 94% on winter days, when 
demand was lower. 

Our analysis also indicated that wind output 
was out of phase with electricity demand during 
certain times of day. For example, during the 
morning hours, around 6:00 a.m., wind output 
usually decreased just as demand was ramping up. 
Throughout the day, demand remained high but 
wind output typically dropped to its lowest level 



Figure 10: Capacity Factors (Expected Output) and 
Capacity Contributions (Output during Peak Electricity 
Demand), by Energy Source (%) 

Source of data: 0PA and IESO 
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for the day. During the evening hours, around 
8:00 p.m., when demand was ramping down, wind 
output was rising, and it remained high overnight 
until early morning. This somewhat inverse rela- 
tionship between daily average wind output and 
daily average demand was particularly pronounced 
in the summer and winter months. 

The OPA has recognized that the lack of correla- 
tion between electricity demand and intermittent 
renewable energy has created operational chal- 
lenges, including power surpluses and the need 
for backup power generated from other energy 
sources. The IESO has been working through its 
Renewable Integration Project to mitigate these 
challenges by engaging stakeholders and establish- 
ing technical working groups to discuss design 
principles, forecasting, and future markets for 
renewable energy. 

Operational Challenge: Surplus Power 

The IESO informed us that increasing the propor- 
tion of renewable energy in the supply mix has 
exacerbated a challenge called surplus base-load 
generation (SBG), a power oversupply that occurs 
when the quantity of electricity from base-load 
generators is greater than demand for electricity. 
Base-load generators are designed to run at a 
steady output 24 hours a day to meet the constant 



2011 Annual Report of the Office of the Auditor General of Ontario 



need or minimum demand for electricity. Ontario's 
base-load fleet includes nuclear units, certain 
hydro stations, and intermittent renewable energy 
sources such as wind. The IESO informed us that 
Ontario did not have any SBG days from 2005 to 
2007, but experienced four such days in 2008, 115 
days in 2009, and 55 days in 2010. The jump in SBG 
days was attributed to several factors, including 
an increase in wind power and a drop in electricity 
demand. 

Given that electricity demand is expected to 
remain relatively flat for at least the next few years 
as more renewable energy comes on-line, there will 
almost certainly be more SBG days in the years to 
come, creating operational challenges and costs that 
will ultimately be borne by electricity ratepayers. 

In 2008, the IESO forecast that, because most 
generators cannot ramp wind power up or down 
in response to demand, SBG hours will increase 
significantly over the next decade. The vast major- 
ity of new renewable energy in the next few years is 
expected to come from wind generators, which typ- 
ically have their highest output overnight and early 
morning, when SBG events are more prevalent. 

Since the prevalence of SBG events could 
threaten the reliability of the electricity system, the 
IESO has been taking action to ease the power sur- 
plus. However, there are technical difficulties and 
cost implications of these actions. Among them: 

• Storing surplus power is difficult because 
of the seasonal nature of renewable energy 
and the need for unrealistically large storage 
capacity. 

• Exporting surplus power is, according to the 
OPA and the IESO, a common and preferred 
way to mitigate power surpluses. Since 2006, 
Ontario has been a net exporter. The IESO 
indicated that although it is difficult to quan- 
tify, the increase in renewable energy has led 
to an increase in exports and put downward 
pressure on export prices. We noted that: 

• In 2010, 86% of wind power was produced 
on days when Ontario was already in a net 
export position. 



• The price Ontarians pay for electricity 
and the price Ontario charges its export 
customers — which are determined by 
the interaction of supply and demand in 
the electricity market — have in recent 
years been moving in opposite directions. 
Although export customers paid only about 
3(t/kWh to 4ct/kWh for Ontario power, 
electricity ratepayers of Ontario paid more 
than 8(t/kWh for this power to be gener- 
ated, as illustrated in Figure 11. 

• Based on our analysis of net exports and 
pricing data from the IESO, we estimated 
that from 2005 to the end of our audit in 
2011, Ontario received $1.8 billion less for 
its electricity exports than what it actually 
cost electricity ratepayers of Ontario. 

• A study in September 2009 also noted that 
Denmark, which relies heavily on wind 
power, has been faced with a similar situa- 
tion and exported large amounts of surplus 
power to Norway and Sweden in order to 
balance domestic supply with demand. 

« Reducing hydro power can be done by 
diverting, or spilling, water from hydro gen- 
erators. The IESO informed us that although 
the magnitude and timing of spill activities 
have not been well documented, Ontario 

Figure 11: Electricity Charge Paid by Ratepayers in 
Ontario vs. Export Price Received by Ontario from 
Other Jurisdictions (<t/kWh) 

Source of data: IESO 
10 
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spilled water to reduce electricity supply on 
96 days in 2009 and 10 days in 2010. Because 
the overall cost to produce hydro power is 
often lower than that of all other types of 
power, reducing hydro power to "make room" 
for wind and solar power is an expensive 
mitigation strategy to reduce surplus power, 
particularly as hydro, wind, and solar power 
are all considered renewable energy sources. 

• Reducing nuclear power is viewed as a last 
resort because nuclear units are designed 
to run constantly and produce at maximum 
capacity. Ramping nuclear units up and 
down involves significant costs and can lead 
to equipment damage. If a nuclear unit is 
shut down, it typically takes 48 to 72 hours 
to restart it. With nuclear energy account- 
ing for the majority of Ontario's electricity, 
such downtime is risky and costly. The IESO 
requested that nuclear generators shut down 
or reduce electricity supply 205 times in 2009 
and 13 times in 2010. 

• Reducing renewable power can be an efficient 
way to reduce supply. Wind generators can be 
brought on-line or off-line quickly — an ideal 
characteristic to address surpluses. Although 
this helps to address the degree to which 

the electricity system is overloaded, it may 
not result in cost savings because if the IESO 
instructs wind generators to shut down under 
a surplus-power situation, the generators 
still get paid under the FIT program (see the 
section titled "FIT Contract Term: Additional 
Contract Payment" earlier in this report) . 

Operational Challenge: Backup Power 
Requirement 

To maintain reliability, there is always a need 
for backup power generation in the event that a 
generator must shut down unexpectedly. However, 
intermittent renewable energy sources such as 
wind and solar require fast-responding backup 
power and/ or storage capacity to keep the supply of 



electricity steady when the skies are cloudy or the 
wind dies down. The OPA informed us that because 
viable large-scale energy storage is not available in 
Ontario, wind and solar power must be backed up 
by other forms of generation. This backup power 
is generated mainly from natural gas, because coal 
will be phased out by the end of 2014. The backup 
requirements have cost and environmental implica- 
tions. For example: 

• The IESO confirmed that consumers have to 
pay twice for intermittent renewable energy — 
once for the cost of constructing renewable 
energy generators and again for the cost of 
constructing backup generation facilities, 
which usually have to keep running at all 
times to be able to quickly ramp up in cases of 
sudden declines in sunlight levels or in wind 
speed. The IESO confirmed that such backups 
add to ongoing operational costs, although no 
cost analysis has been done. 

• The use of gas-fired backup generation will 
reduce the net contribution of renewable 
energy to environmental protection, as indi- 
cated by studies from other jurisdictions (see 
the "Environmental and Health Impacts" sec- 
tion later in this report) . 

Despite these concerns, the cost and environ- 
mental impacts of such backup generation capacity 
were not formally analyzed to ensure that this 
information would be available to policy decision- 
makers. We noted that: 

• Prior to the passage of the Green Energy and 
Green Economy Act in 2009, the Ministry did 
not quantify how much backup power would 
be required. It was not until February 2011 
that the Minister issued a new supply-mix dir- 
ective that asked the OPA to consider backup 
options, such as converting coal-fired plants 
to gas-fired operation, importing power from 
other jurisdictions, and developing storage 
systems. The OPA has not yet made any rec- 
ommendations to the Ministry. 

• The only analysis on backup power that the 
Ministry cited was a study done by a third 
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party engaged by the OPA as part of its 2007 
IPSP development. The study noted that 
10,000 MW of wind would require an extra 
47% of non-wind sources to handle extreme 
drops in wind. We noted that the third party 
who carried out this study also operated an 
Ontario wind farm, raising questions about 
the study's objectivity. In spite of this, the OPA 
and the Ministry did not confirm or update 
this study's projections and did not determine 
how much backup power would be required. 
According to the OPA, a new IPSP will assess 
the operational challenges of surplus power and 
backup requirements. At the time of our audit, the 
new IPSP was still under development. 



RECOMMENDATION 5 



To ensure that the stability and reliability of 
Ontario's electricity system is not significantly 
affected by the substantial increase in renew- 
able energy generation over the next few years, 
the Ontario Power Authority should continue to 
work with the Independent Electricity System 
Operator to assess the operational challenges 
and the feasibility of adding more intermittent 
renewable energy into the system, and advise 
the government to adjust the supply mix and 
energy plan accordingly. 



MINISTRY RESPONSE 



The Ministry agrees that system reliability and 
stability is a key element in energy system plan- 
ning. The Ministry will work collaboratively 
with the IESO, the OPA, and all partners in the 
sector to ensure that the system is capable of 
meeting new challenges. 

Ontario, as part of the North America-wide 
interconnected network, is required to plan 
for an agreed-to level of reliability, which is 
developed and monitored by the North Amer- 
ican Electric Reliability Corporation. A focus 
of this requirement is on the ability to reliably 
meet annual peak electricity demand. A system 



that fails to do so would create reliability risks 
with other interconnected systems. 

We note that the increases in renewable 
energy generation do not increase greenhouse 
gas emissions. Without renewable energy gen- 
eration, the gas-fired generation would have to 
run more frequently, resulting in higher green- 
house gas emissions. 

| OPA RESPONSE 

The OPA agrees with the recommendation and 
is working with the IESO to improve the integra- 
tion of renewable energy and to explore how 
changes to the supply mix and to contractual 
requirements could maximize the benefits 
of intermittent generators for the Ontario 
electricity grid and ratepayers. The OPA will 
continue to provide advice for the government's 
consideration in determining the supply mix. 
Ongoing planning has already contributed to 
greater understanding of the issues and solu- 
tions required to integrate renewable energy. 



DELIVERY OF RENEWABLE ENERGY 

As a result of the Green Energy and Green Economy 
Act, 2009 and the FIT program, there has been 
enormous demand for connecting renewable 
energy to Ontario's electricity grid. As a result, 
additional transmission and distribution develop- 
ments are required to facilitate the connection and 
delivery of renewable energy resources. 

Impact of Renewable Energy on 
Transmission and Distribution Systems 

Because the FIT program has created many new 
points of generation, especially in northern 
Ontario, significant investments are required to 
update and expand transmission and distribution 
systems to get the electricity from numerous remote 
and widely dispersed renewable energy generators 
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to population centres in southern Ontario. Costs 
associated with these investments are paid by elec- 
tricity ratepayers through increases in the delivery 
charges on electricity bills. Specifically: 

• The Ministry's Long-Term Energy Plan 
identified five priority transmission projects, 
including three designed to accommodate 
renewable energy, at an estimated total cost 
of about $2 billion. According to the OPA, 
the three priority projects were intended to 
accommodate 1,900 MW of renewable energy 
at an estimated cost of between $450 million 
and $850 million, and also to contribute 

to system reliability and increase transmis- 
sion capability. Hydro One indicated that 
the actual timing and cost of these priority 
projects is uncertain, because they depend on 
complex and often lengthy approval processes 
by the OEB, the Ministry of the Environment, 
and others. There may also be unexpected 
capital expenditures due to unforeseen 
technical problems, because new technology 
is required for transmission and distribution 
systems to support renewable energy. 

• In addition to the three priority projects, the 
Bruce-Milton line is expected to go into ser- 
vice in December 2012 to deliver 1,500 MW 
of nuclear power and 1,700 MW of renewable 
energy in southern Ontario. The cost of this 
line was initially estimated at $635 million, 
but the estimate was raised in March 2011 

to $755 million. Hydro One attributed the 
$120-million cost overrun to delays in pro- 
ject approvals and higher-than-anticipated 
labour and material costs. The overrun could 
increase further by the time Bruce-Milton is 
complete. The three other priority projects 
could face similar cost overruns if similar 
labour and material cost pressures arise. 

• Hydro One files applications with the OEB 
to seek approval to recover the costs of 
transmission and distribution charges on 
electricity bills. Its most recent distribution 
rate application estimated that investments of 



$169 million in 2010 and $296 million in 2011 
would need to be recovered from electricity 
ratepayers for the cost of connecting renew- 
able energy to the distribution systems and 
modernizing the electricity grid. 
Apart from the cost implications, the OPA was 
aware that only limited capacity was readily avail- 
able to FIT when the program was launched. To 
date, Ontario's existing transmission and distribu- 
tion systems have already been operating at or 
near capacity, but there has been a higher-than- 
anticipated number of FIT projects attempting to 
connect into the system. The capacity limitation 
has hindered the timely connection of renewable 
energy to the grid and kept the FIT program from 
achieving its full potential. 

As of April 1, 2011, more than 3,000 FIT applica- 
tions with a total capacity of about 10,400 MW 
could not be accommodated by the existing 
transmission infrastructure and were awaiting con- 
nection. Of the 10,400 MW awaiting connection, 
only about 2,400 MW will be accommodated by the 
future transmission capacity of the Bruce-Milton 
line and the three other priority projects. The 
remaining 8,000 MW will not be connected unless 
new lines are built or existing ones upgraded. Most 
of this is from FIT applications prior to June 2010, 
and these have been awaiting an Economic Connec- 
tion Test (ECT) to determine whether it is econom- 
ical to build additional transmission infrastructure. 
Therefore, connecting renewable energy projects to 
the grid is subject to both technical and economic 
considerations, and there is no guarantee that 
every project will be connected. However, the Min- 
istry informed us that the requirement to conduct 
the ECT process was superseded by the Long-Term 
Energy Plan (LTEP) in November 2010. Therefore, 
as of April 2011, the OPA had not yet started the 
first ECT, which was to have been conducted in 
August 2010 and every six months thereafter on a 
rotating basis. 
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Allocation of Capacity to Korean 
Consortium 

As noted earlier, the Ministry signed an agreement 
with a consortium of Korean companies that agreed 
to develop 2,500 MW of renewable energy resour- 
ces in Ontario in five phases by 2016. Besides pay- 
ing the consortium contract prices higher than the 
standard FIT prices if it meets its job-creation tar- 
gets, another aspect of the consortium agreement 
is its impact on transmission capacity for other 
renewable energy projects. In April 2010, the Min- 
ister directed the OPA to give priority to connecting 
the consortium projects to the grid when assessing 
the availability of already-limited transmission cap- 
acity. This commitment to the consortium affected 
the FIT contract allocation process and the timely 
connection of renewable energy from other gener- 
ators. Specifically: 

• When the OPA evaluated the FIT applications 
and the availability of transmission capacity, 
it had to consider the locations and sizes of 
the consortium projects and their transmis- 
sion requirements. According to the OPA, 
the required Economic Connection Test was 
delayed because the OPA could not start to 
assess the transmission availability until the 
consortium finalized the connection points for 
phases two and three of its projects. 

• Two of the three priority transmission pro- 
jects were selected partly because they were 
expected to meet the timing requirements of 
the consortium agreement. Specifically, the 
OPAs forecasts of the likely locations of the 
consortium projects indicated that 1,323 MW 
of the existing transmission capacity and 
about 1,177 MW of the future transmission 
capacity from the Bruce-Milton line and the 
other three priority projects will be made 
available to the consortium. 



Planning of Transmission Systems 

Planning and co-ordinating the timelines of 
transmission development is not unique to the FIT 
program; its open nature, however, has created 
uncertainties and challenges for the OPA. 

The OPA can identify the capacity and con- 
necting points of renewable energy generators as 
well as the future needs and locations of transmis- 
sion lines only after it receives the FIT applica- 
tions. The OPA noted that this has created a new 
challenge, which it has dubbed "chicken and egg": 
transmission capacity requirements cannot be 
known in the absence of renewable energy gener- 
ators, and renewable energy generators cannot go 
forward in the absence of transmission capacity. 
In essence, new transmission projects cannot be 
built unless there are proven needs and firm com- 
mitments from renewable energy developers, but 
renewable energy developers are not willing to 
invest money to build generators without the pres- 
ence of adequate transmission capacity because 
of the risk that they will not be connected to the 
grid. This situation will affect the timeliness of 
connecting renewable energy to the system because 
the lead time for transmission projects, about five to 
seven years, is much longer than the two-to-three- 
year lead time for renewable energy projects. 



RECOMMENDATION 6 



To provide investors who have submitted 
applications for Feed-in Tariff (FIT) projects 
with timely decisions on whether their projects 
can be connected to the grid and to ensure that 
adequate transmission capacity is available for 
approved projects, the Ontario Power Authority 
should work with the Ministry of Energy and 
Hydro One to: 

• identify practical ways to deal on a timely 
basis with the FIT investors who have been 
put on hold; and 

• prioritize the connection of approved FIT 
projects to the grid. 



Electricity Sector— Renewable Energy Initiatives 




The Ministry continues to work closely with 
the OPA, Hydro One, and local distribution 
companies to improve connection access for 
FIT and microFIT projects. 

The province's Long-Term Energy Plan 
identifies five priority transmission projects, 
which have been identified in large part on the 
basis of their ability to allow greater renewable 
connection. 

Recently, the Minister of Energy asked Hydro 
One to expedite infrastructure upgrades for up 
to 15 of the most severely constrained hydro 
transformer stations to enable the connection 
of more microFIT projects. The Minister also 
issued a directive to the OPA in August 2011 
directing the OPA to provide connection options 
to constrained microFIT proponents. 

In addition, working to prioritize and effect- 
ively connect FIT and microFIT projects will be 
a key focus of the two-year review of the FIT 
program. 




The OPA agrees with this recommendation. 
The OPA has continued to work closely with the 
Ministry and Hydro One to improve connection 
access for FIT and microFIT projects. In August 
2011, for example, the OPA began to implement 
a ministerial directive that allows microFIT pro- 
ponents to select from various options to relocate 
constrained projects to areas where connection 
is possible. Prior to developing the FIT program, 
the Renewable Energy Supply Integration Team 
was established by the OPA, the Ontario Energy 
Board, and Hydro One to provide advice and 
co-ordinate and streamline activities related to 
the expansion of renewable energy, including 
connecting renewable generators to the trans- 
mission and distribution systems. The OPA will 
continue to work with sector partners and the 
Ministry on connection issues. 



SOCIO-ECONOMIC, ENVIRONMENTAL, 
AND HEALTH IMPACTS OF RENEWABLE 
ENERGY 

Socio-economic Impacts 

The Green Energy and Green Economy Act, 2009 
(Act) was intended to support new investment and 
economic growth in Ontario through the creation 
of a strong and viable renewable energy sector. 

Job Creation in Ontario 

The Ministry said the Act is expected to support over 
50,000 direct and indirect jobs over three years in 
transmission and distribution upgrades, renewable 
energy, and conservation. We questioned whether 
the job projection information was presented as 
transparently as possible. For example: 

• A majority of the jobs will be temporary. The 
Ministry projected that of the 50,000 jobs, 
about 40,000 would be related to renew- 
able energy. Our review of this projection 
suggests that 30,000, or 75%, of these jobs 
would be construction jobs and would last 
only from one to three years, while the 
remaining 10,000 would be long-term jobs in 
manufacturing, operations, maintenance, and 
engineering. However, the high proportion 
of short-term jobs was not apparent from the 
Ministry's public announcement. 

• The 50,000-job projection included new 
jobs but not those jobs that would be lost 
as a result of promoting renewable energy. 
Experience in other jurisdictions suggests 
that jobs created in the renewable energy 
sector are often offset by jobs lost as a result 
of the impact of higher renewable energy 
electricity prices on business, industry, and 
consumers, as indicated in Figure 4. In addi- 
tion, the closure of Ontario's coal-fired plants 
by the end of 2014 will lead to job losses, but 
these were not factored into the Ministry's job 
projections. Ontario Power Generation, which 
operates the coal-fired plants, informed us 
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that the extent of job losses depended on the 
Ministry's plan: about 2,300 jobs would be 
lost if the Ministry closed all coal-fired plants, 
but 600 of these could be saved if certain 
coal-fired plants are converted to biomass or 
gas-fired operation. The Ministry's Long-Term 
Energy Plan noted that Ontario will continue 
to explore the opportunities for using biomass 
along with natural gas in the coal-fired plants. 

Experiences in Other Jurisdictions 
We noted that Ontario's job projections were 
not consistent with the experiences of other 
jurisdictions that have a longer history with 
renewable energy. Studies from these countries 
highlighted issues with renewable energy that 
included job losses and high costs per "green" 
job. We questioned whether the experiences of 
other jurisdictions had been taken into considera- 
tion, and the Ministry confirmed that it had not 
estimated the potential job losses and the cost 
per renewable-energy-related job in Ontario. In 
particular, Ontario's FIT program was modelled on 
the FIT programs in Germany and Spain, and their 
job-related experiences could well be relevant to 
Ontario. For example, we noted the following stud- 
ies conducted over the past three years: 

• A 2009 study conducted in Germany noted 
that job projections in the renewable energy 
sector conveyed impressive prospects of 
gross job growth but omitted such offsetting 
impacts as jobs lost in other energy sectors 
and the drain on economic activity caused by 
higher electricity prices. The study found that 
the cost of creating renewable-energy-related 
jobs was up to US$240,000 per job per year, 
far exceeding average wages in other sectors. 

• A 2009 study conducted in Spain found 
that for each job created through renewable 
energy programs, about two jobs were lost in 
other sectors of the economy. 

A 2009 study conducted in Denmark noted 
that a job created in the renewable sector does 



not amount to a new job but, rather, usually 
comes at the expense of a job lost in another 
sector. The study also found that each job 
created under renewable energy policies cost 
between US$90,000 and US$140,000 per year 
in public subsidies — or about 175% to 250% 
of the average wage paid to manufacturing 
workers in Denmark. 
• A 2011 study conducted in the United King- 
dom (after the FIT program was launched in 
Ontario) reported that about four jobs were 
lost elsewhere in the economy for every one 
new job in the renewable energy sector, pri- 
marily because of higher electricity prices. 
In November 2010, similar concerns were raised 
about the Ontario job projections in a report by 
the Task Force on Competitiveness, Productivity 
and Economic Progress of the Rotman School of 
Management at the University of Toronto. The 
report noted that it is unclear what the jobs estimate 
includes, because it has offered neither a definition 
of green jobs nor a transparent calculation of how 
the 50,000 figure was arrived at. The report also 
said that it is unclear whether the 50,000 estimate 
is a gross or net number of jobs. The report further 
noted that even if 50,000 new jobs were created, 
the higher energy costs attributable to renewable 
energy might result in job losses elsewhere in the 
economy, particularly in industries that use large 
quantities of energy. Another recent study in Can- 
ada estimated that each new job to be created as 
a result of renewable energy programs would cost 
$179,000 per year. 



RECOMMENDATION 7 



To ensure that the provincially reported estimate 
of jobs created through the implementation of 
the renewable energy strategy is as objective and 
transparent as possible, the analysis should give 
adequate consideration to both job-creation and 
job-loss impacts, as well as job-related experien- 
ces of other jurisdictions that have implemented 
similar renewable energy initiatives. 
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The Ministry's calculation of 50,000 jobs relied 
on standard Ontario government methodology, 
including standard investment and job multipli- 
ers. The figure of 50,000 jobs has always been 
characterized by the Ministry as a mix of long- 
term and short-term jobs. 

Lessons learned from other jurisdictions 
with respect to job-creation and job-loss impacts 
will be taken into account where they may be 
comparable or instructive to Ontario, taking 
into account the fact that renewable-energy- 
program administration rules vary, as does the 
composition of the economies. 



Environmental and Health Impacts of 
Renewable Energy 

Ontario's 2007 Climate Change Action Plan 
outlined "coal phase-out, renewables, and other 
electricity initiatives" as measures to help Ontario 
achieve its greenhouse gas reduction targets, which 
call for reductions below 1990 levels of 6% by 2014, 
15% by 2020, and 80% by 2050. 

The Ministry's 2010 Long-Term Energy Plan 
reiterated the commitment to improve the health of 
Ontarians and to fight climate change by investing 
in renewable energy and phasing out coal, which is 
the largest source of greenhouse gases and accounts 
for a number of health and environmental problems. 

Environmental Concerns 
The Ministry indicated that renewable energy will 
help reduce greenhouse gases by displacing gas- 
fired generation. However, as noted earlier, any sig- 
nificant increase in intermittent renewable energy 
requires backup power by either coal- or gas-fired 
plants because wind and solar power have relatively 
low reliability and capacity. In Ontario's case, 
because coal-fired plants are being phased out by 
the end of 2014, this backup will need to come from 



gas-fired plants. Although gas-fired plants emit 
fewer greenhouse gases than coal-fired plants, they 
still contribute to greenhouse gas emissions. Our 
review of experiences in other jurisdictions showed 
that the original estimated reduction in greenhouse 
gases had not been reduced to take into account the 
continuing need to run fossil-fuel backup power- 
generating facilities. For instance: 

• A 2008 study in the United Kingdom found 
that power swings from intermittent wind 
generation need to be compensated for by 
natural-gas generation, which has meant less 
of a reduction in greenhouse gases than ori- 
ginally expected. 

• A 2009 study in Denmark noted that although 
the country is the world's biggest user of wind 
energy, it has had to keep its coal-fired plants 
running to maintain system stability. 

• The German government also had to build 
new coal-fired plants and refurbish old ones to 
cover electricity requirements that could not 
be met through intermittent wind generation. 

According to the Ministry, Ontario is unique 
in its commitment to phase out coal by the end of 
2014: other jurisdictions did not make that com- 
mitment. The Ministry has not yet quantified how 
much backup power will be required from other 
energy sources to compensate for the intermittent 
nature of renewable energy, and accordingly has no 
data on the impact of gas-fired backup power plants 
on greenhouse gas emissions. 

Health Concerns 
In recent years, there have been growing public- 
health concerns about wind turbines, particularly 
with regard to the noise experienced by people liv- 
ing near wind farms. In May 2010, Ontario's Chief 
Medical Officer of Health issued a report conclud- 
ing that available scientific evidence to date did not 
demonstrate a direct causal link between wind tur- 
bine noise and adverse health effects. However, the 
report was questioned by environmental groups, 
physicians, engineers, and other professionals, who 
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noted that it was merely a literature review that 
presented no original research and did not reflect 
the situation in Ontario. We also noted that only a 
limited number of renewable generators were in 
operation in Ontario when the report was prepared 
in spring 2010, a few months after the launch of the 
FIT program. 

One of the provisions of the Act was the estab- 
lishment of an academic research chair to examine 
the potential effects of renewable energy generators 
on public health. In February 2010, an engineer- 
ing professor from the University of Waterloo was 
appointed to this position but, as of July 2011, there 
had been no report on the results of any research 
conducted to date. 



RECOMMENDATION 8 



To ensure that renewable energy initiatives are 
effective in protecting the environment while 
having minimal adverse health effects on indi- 
viduals, the Ministry of Energy should: 

• develop adequate procedures for tracking 
and measuring the effectiveness of renew- 
able energy initiatives, including the impact 
of backup generating facilities, in reducing 
greenhouse gases; and 

• provide the public with the results of object- 
ive research on the potential health effects of 
renewable wind power. 



| MINISTRY RESPONSE 




The Ministry agrees that the impacts of increas- 
ing the share of renewable energy in Ontario's 
energy mix should be quantified where pos- 
sible and underpinned by objective research. 
For example, a 2005 independent study, Cost 



Benefit Analysis: Replacing Ontario's Coal-Fired 
Electricity Generation, found that if health and 
environmental impacts were accounted for, 
the total cost of coal-fired generation would be 
$4.4 billion per year. This study helped reaffirm 
the province's decision to phase out coal and 
to increase the share of renewable energy in 
Ontario's energy mix. 

The Ministry will continue to rely on the 
Chief Medical Officer of Health to provide 
objective advice on the potential health impacts 
of renewable energy generators. The Chief Med- 
ical Officer of Health's recent review found that 
the scientific evidence does not demonstrate any 
direct causal link between wind turbine noise 
and adverse health effects. 

The Ministry will continue to work with 
other ministries to promote further scientifically 
based information about the impacts of renew- 
able energy. For example, the Ministry of the 
Environment has appointed an independent 
research chair for a five-year term to undertake 
research on the health impacts of renewable 
energy generators. Considerable work is well 
under way by the chair and his team to address 
the important technological, health, and safety 
aspects of the renewable energy technologies. 



| 0PA RESPONSE 




Ongoing plans, including the Integrated Power 
System Plan, identify the environmental emis- 
sions from planned resources, and they clearly 
identify a reduction in emissions over the time 
that the OPA has been involved in planning and 
procuring resources and through the planning 
horizon. 



Chapter 3 



Ministry of Finance 

Section 

■ 3.04 Electricity Sector 

Stranded Debt 



Background 



We provided updates in past Annual Reports on 
the electricity sector's stranded debt, denned as 
that portion of the total debt of the old Ontario 
Hydro that could not be serviced in a competitive 
market environment when the electricity sector 
was restructured 12 years ago. We provide another 
such update this year, along with information on 
the Debt Retirement Charge (DRC), a component of 
nearly every Ontario ratepayer's electricity bill. 




HOW DID THE STRANDED DEBT ARISE? 

With passage of the Energy Competition Act, 1998 
(which included the Ontario Energy Board Act, 
1998 and the Electricity Act, 1998), the Ontario 
government launched a major restructuring of the 
province's electricity industry. 

The two most significant aspects of this restruc- 
turing were the creation of competitive wholesale 
and retail markets for electricity that opened May I, 
2002, and the breakup of Ontario Hydro into five 



successor companies on April 1, 1999. The new 
entities and their functions were: 

• Ontario Power Generation (OPG) for electri- 
city generation; 

• Hydro One for wholesale power transmission 
and retail distribution; 

• the Electrical Safety Authority for electrical 
inspection; 

• the Independent Electricity Market Operator 
to manage the power grid and the wholesale 
electricity market; and 

• the Ontario Electricity Financial Corporation 
(OEFC) to manage the legacy debt and other 
liabilities not transferred from the old Ontario 
Hydro to successor companies. 

The intent of restructuring was to have the new 
entities operate in a commercial manner, with a 
strong financial footing that would make them 
more efficient and effective and lead to prices that 
were as low as possible for consumers. 

Under the old monopolistic rate-regulated 
system, Ontario Hydro tried to set electricity prices 
at a level that ensured all costs, including principal 
and interest payments on debt, were eventually 
recovered from customers. With the introduction of 
competition, however, Ontario Hydro's commercial 
successor companies no longer have the assurance 
that they can recover all costs, especially those 
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incurred in previous years and still outstanding, 
through free-market electricity rates. 

One of the most critical steps in the restructur- 
ing process was to determine the fair market value 
of the Ontario Hydro assets transferred to the new 
entities. Both Ontario Hydro and the government, 
assisted by private-sector investment firms and 
other experts, recognized that the market value of 
these assets in a competitive environment would 
be significantly less than the amounts recorded in 
Ontario Hydro's books. It was anticipated that in a 
competitive market, revenues and profits generated 
by the successor companies would not be sufficient 
either to justify the existing recorded asset values or 
to service Ontario Hydro's substantial outstanding 
debt. 

On April 1, 1999, the Ministry of Finance 
determined that Ontario Hydro's total debt and 
other liabilities stood at $38.1 billion, which greatly 
exceeded the estimated $17.2-billion market value 
of the assets being transferred to the new entities. 
The resulting shortfall of $20.9 billion was deter- 
mined to be "stranded debt," representing the total 
debt and other liabilities of Ontario Hydro that 
could not be serviced in a competitive environment. 

Responsibility for servicing and managing the 
legacy debt of Ontario Hydro, which includes 
the stranded debt, was given to the OEFC, whose 
opening balance sheet reflected a stranded debt 
or unfunded liability of $19.4 billion. This 
amount represented the difference between the 
$18.7-billion value of assets assumed by the OEFC 
and the $38.1 billion of Ontario Hydro legacy debt 
and other liabilities that the OEFC also took on. It 
should be noted that the stranded-debt figure of 
$19.4 billion listed as an unfunded liability in the 
OEFC's April 1, 1999, financial statements does not 
agree exacdy with the total of $20.9 billion in Figure 1 
because of certain accounting adjustments, consisting 
mainly of $1.2 billion in deferred debt costs. 

Because the OEFC had neither the assets nor 
the expected revenue streams from the value of the 
other Ontario Hydro successor companies to fully 
service the debt obligations it had assumed, the 



Figure 1: Financial Impact of the Restructuring of the 
Electricity Sector ($ billion) 

Source of data: Ministry of Finance 

Former Ontario Hydro debt and liabilities 38.1 
less 

Value of new generation and service companies 17.2 
equals 

Stranded debt 20.9 
less 

Expected dedicated revenue streams 13.1 
equals 

Residual stranded debt 7.8 



Electricity Act provided for other revenue streams, 
and the government developed a long-term plan to 
provide the OEFC with revenue streams to service 
the existing debt and ultimately retire the stranded 
debt. A key government policy decision at the time 
was that all electricity-sector debt would be repaid 
by the electricity sector and ratepayers rather than 
from general tax revenues. 

To service and retire the $20.9 billion in 
stranded debt, the government established a long- 
term plan wherein the burden of debt repayment 
would be borne partly through dedicated revenues 
from the electricity-sector companies — OPG, Hydro 
One, and Municipal Electrical Utilities (MEUs) — 
and partly by electricity consumers. This would be 
broken down for the electricity sector as follows: 

• The electricity companies would make pay- 
ments in lieu of taxes (PILs) to the OEFC. 
PILs are equivalent to the corporate income, 
property, and capital taxes paid by private 
corporations. Before April 1, 1999, these com- 
panies were not required to make any of these 
payments. 

• The cumulative annual combined profits of 
OPG and Hydro One in excess of the govern- 
ment's $520-million annual interest cost of its 
investments in the two companies would go 
toward repaying stranded debt. 

As of April 1, 1999, the present value of these 
two dedicated revenue streams was estimated at 
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$13.1 billion. The estimated balance remaining 
on the $20.9-billion stranded debt, amounting 
to $7.8 billion, was called the "residual stranded 
debt," and the Electricity Act provided for a new 
Debt Retirement Charge (DRC) to be paid by elec- 
tricity consumers until the residual stranded debt 
was retired. In essence, this was the estimated por- 
tion of the stranded debt that could not be serviced 
by the estimated present value of the two dedicated 
revenue streams from the electricity companies. 

This structure was intended to achieve the 
elimination of the stranded debt in a prudent man- 
ner, and to distribute the burden of debt repayment 
between electricity consumers and the electricity 
sector (OPG, Hydro One, and MEUs). 

Figure 1 illustrates the significant determina- 
tions made at the time of restructuring. 

PROGRESS IN RETIRING THE STRANDED 
DEBT 

Figure 2 illustrates the progress made in reducing 
the total stranded debt as reflected in the OEFC's 
audited financial statements. It should be noted 
that the OEFC's financial statements currently 
report only the total outstanding stranded debt and 
not the residual stranded debt, which is a compon- 
ent of total stranded debt. This is consistent with 
the Electricity Act, 1998, which requires that the 
outstanding amount of the residual stranded debt 
need only be determined and disclosed "from time 
to time," as discussed later in this section. 

Progress in retiring the overall stranded debt 
over this period has been slower than anticipated, 
due primarily to the lower-than-expected profitabil- 
ity of Hydro One and, particularly, OPG. The lower 
their respective earnings, the lower the PILs they 
make to the OEFC, and the less they contribute to 
the electricity-sector dedicated income payments. 
The uncertainty inherent in forecasting the finan- 
cial performance of the electricity sector becomes 
apparent on examination of the actual revenues 
produced by this sector. As reported in the OEFC's 
audited financial statements, PILs revenues have 



Figure 2: Progress in Repayment of the Stranded Debt, 
1999/2000-2010/11 ($ billion) 

Source of data: OEFC 



1 Fiscal Year End 




at April 1, 1999 


19.4 


1999/2000 


20.0 


2000/01 


20.0 


2001/02 


20.1 


2002/03 


20.2 


2003/04 


20.6 


2004/05 


20.4 


2005/06 


19.3 


2006/07 


18.3 


2007/08 


17.2 


2008/09 


16.2 


2009/10 


14.8 


2010/11 


13.4 



varied from a low of $321 million in the 2010/11 fis- 
cal year to a high of $949 million in 2005/06, while 
electricity-sector dedicated income has ranged from 
$0 in 2001/02, 2003/04, and 2008/09 to a high of 
$771 million in 2010/11. 

As we reported in last year's Annual Report, 
some of the factors that have affected OPG's profit- 
ability over the past 11 years include the following: 

• Electricity-generation projects such as the 
Niagara tunnel project have had cost over- 
runs, with costs being almost double their ori- 
ginal estimates and completion dates running 
years behind schedule. 

• Investment returns on the $ll-billion nuclear 
removal and waste management funds, 
reflected in OPG's financial statements, have 
been volatile. Canadian accounting standards 
require OPG to reflect unrealized gains and 
losses in its net income. 

• There has been public and political pressure to 
keep electricity rates low, which impacts the 
profitability of the sector. 

As well, OPG's future profitability and, 
consequently, the pace of the reduction of the 
recorded stranded-debt balance may be impacted 
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by uncertainty about which costs the Ontario 
Energy Board (OEB), which regulates rates, will 
allow OPG to pass on to its customers. In a recent 
application, for example, OPG sought a 6.2% rate 
increase for power produced by its major generat- 
ing stations. However, the OEB granted just 1%. At 
the time of this writing, OPG was appealing parts 
of the OEB decision, but should it lose the appeal 
and be unable to reduce costs, OPG will experience 
lower profitability. 

THE RESIDUAL STRANDED DEBT AND 
THE DEBT RETIREMENT CHARGE (DRC) 

In recent months, issues surrounding the electricity 
Debt Retirement Charge (DRC) have been raised 
both in the Legislature and in the media. These 
issued have included questions such as: 

• Are there any restrictions on what the DRC 
can be used for? 

• How is the residual stranded debt 
determined? 

• How long will the DRC remain on electricity 
bills? 

Accordingly, we decided to incorporate a review 
of the DRC in this year's update. Because charges 
of this nature typically need to have underlying 
legislation or regulations that authorize their col- 
lection, a key aspect of our work was a review of the 
underlying legislation with respect to the levying 
and use of the DRC. We also wanted to provide the 
Legislature and the public with information that 
could help answer the above questions. 

What Can the DRC Be Used For? 

Collection of the DRC began on May 1, 2002, the 
day the electricity market opened to competition. 
The rate was established at 0.7$ per kilowatt hour 
(kWh) of electricity and remains the same today. 
Currently, the OEFC collects more than $900 mil- 
lion a year in DRC revenue, and, as of March 31, 
2011, approximately $8.7 billion in DRC revenue 
had been collected. 



In announcing the DRC in 2000, the then Minis- 
ter of Energy said that the objective of it was to pay 
down the estimated $7.8-billion residual stranded 
debt: "All revenues from the DRC will go directly 
to the Ontario Electricity Financial Corporation to 
be used exclusively to service the residual stranded 
debt. Once the residual stranded debt has been 
retired, the DRC will end." 

Although this reference implies that the DRC 
must be used only to pay down the residual 
stranded debt, it is the underlying legislation that 
dictates the collection and usage of the DRC and 
that drives the actions of both the OEFC and the 
responsible Minister with regard to compliance. 

Section 85 of the Electricity Act, 1998 (Act), 
which is titled "The Residual Stranded Debt and 
the Debt Retirement Charge," gives the government 
the authority to implement the DRC, and this same 
section specifies when it is to end. The OEFC's 2010 
Annual Report states that the Act "provides for the 
DRC to be paid by consumers until the residual 
stranded debt is retired. The debt repayment 
plan estimates residual stranded debt will likely 
be retired between 2015 and 2018"; and its 2011 
Annual Report states that the OEFC "receives the 
Debt Retirement Charge paid by electricity consum- 
ers at a rate of 0.7 cents/kWh until the residual 
stranded debt is retired." Given that the wording 
in section 85 was, in our view, open to interpreta- 
tion in several areas, we engaged external legal 
counsel to assist us. The following comments and 
observations take into consideration the opinion 
of our external legal advisers with respect to the 
interpretation of this section. 

Although DRC funds collected are separately 
disclosed in the OEFC's financial statements, they 
are not segregated or separately accounted for 
with respect to the residual stranded debt, but are 
combined with the OEFC's other revenue sources 
and used for general corporate purposes. Our view, 
which is supported by legal advice, is that section 
85 does allow DRC funds to be used for any purpose 
that is in accordance with the objectives and pur- 
poses of the OEFC. In essence, and notwithstanding 
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the comments made by the then Minister when 
tabling the original legislation, section 85 does not 
restrict the application of the DRC to being used 
only to service the outstanding residual stranded 
debt. This means that the OEFC, in comingling the 
DRC funds collected with its other sources of rev- 
enue and in using all funds collected to carry out its 
legislated objects, has complied with the underlying 
legislation with respect to the DRC. 

However, as discussed later in this section, we 
believe that the Minister does have certain addi- 
tional responsibilities under section 85 with respect 
to when the collection of the DRC will end. 

How Is the Outstanding Residual Stranded 
Debt Determined? 

The Act does not specify precisely how the deter- 
mination or calculation of the outstanding amount 
of the residual stranded debt is to be done, only 
that it be done "from time to time." However, the 
Act states that the government, by regulation, may 
establish what is to be included in the calculation of 
the residual stranded debt. As yet, though, no such 
regulation has been made. With respect to how to 
calculate the outstanding residual stranded debt at 
any one time, we believe that there are several pos- 
sible approaches, including the following: 

• Have the initial $7.8-billion residual stranded 
debt less the DRC collected to date equal the 
remaining residual stranded debt. 

• Treat the initial $7.8-billion debt like a mort- 
gage or loan, in which each DRC payment 
would go to paying interest and paying down 
the principal. 

• Apply DRC revenues to the payment of a 
portion of the OEFC's operating expenses, 
in addition to interest and financing costs, 
because there are administrative costs associ- 
ated with running the OEFC. 

• Consider the residual stranded debt to be 
retired only when the OEFC's assets, which 
include the present value of projected revenue 



streams from the electricity sector, equal its 
liabilities. 

The last approach was essentially the method 
used in the original 1999 calculation of the esti- 
mated stranded debt and residual stranded debt. 
We understand that it has continued to be used 
by the OEFC since that time. In effect, under this 
methodology, low past profitability or estimated 
future profitability in the electricity sector would 
result in more of the stranded debt being paid by 
electricity consumers through DRC payments, 
whereas high past profitability or estimated future 
profitability would reduce the amount of DRC pay- 
ments needed from consumers. 

Although the definition of residual stranded 
debt in section 85 does not include interest or other 
OEFC expenses, and notwithstanding the lack of 
specific statutory guidance as to how the residual 
stranded debt is to be calculated, we believe that 
the legislation does provide the Minister with a fair 
degree of latitude in determining how the residual 
stranded debt is to be calculated and reported. The 
legislation also allows, but does not require, the 
government to make regulations governing deter- 
minations of the stranded debt and the residual 
stranded debt for the purposes of section 85. 

When Does Collection of the DRC End? 

Section 85 requires that the Minister of Finance 
determine the total stranded debt, determine "from 
time to time" the residual stranded debt, and make 
these determinations public. When the Minister 
determines that the residual stranded debt has 
been retired, collection of the DRC must cease. 
Since passage of the Act over a decade ago, the 
Minister has made no such public determination of 
the outstanding amount of the residual stranded 
debt, and the DRC has continued to be collected. 

From our perspective, the key question is 
what "from time to time" means. Can it be totally 
open-ended and left solely up to the discretion 
of the government of the day as to when such a 
determination will be made and the DRC collected 
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indefinitely until that time? Or, if not, what would 
be a reasonable time frame within which the Minis- 
ter should make a determination of the outstanding 
amount of the residual stranded debt to meet the 
requirements of section 85? 

Our view is that it is certainly reasonable for the 
Minister not to have made a determination of the 
outstanding amount of the residual stranded debt 
obligation in the initial years after collection of the 
DRC began. However, we believe that the intent 
of section 85 is that the Minister must provide a 
periodic update to the consumers paying the DRC 
with respect to the status of this particular charge 
on their electricity bills. 

Given that the DRC has been collected from 
electricity consumers for almost a decade and 
that more than $8 billion in DRC revenue has 
been collected during that time, our view is that 
the Minister should make a formal determina- 
tion of the outstanding amount of the residual 
stranded debt in the near future and make this 
determination public. Consideration should also 
be given to that part of section 85 that allows the 
government to establish and clarify, by regulation, 
when such a determination will be made and how 
the amount of the outstanding residual stranded 
debt is to be calculated. 

| MINISTRY OF FINANCE RESPONSE 

The Ministry of Finance (Ministry) concurs with 
the Auditor's report with respect to the Ontario 



Electricity Financial Corporation (OEFC) being 
in compliance with the Electricity Act, 1998 
(Act) in the use of Debt Retirement Charge 
(DRC) revenues. DRC revenues are used by the 
OEFC to perform its objectives under the Act, 
including servicing and retiring its debt and 
other liabilities. The OEFC's expenses included 
interest payments of about $1.6 billion in the 
2010/11 fiscal year. Since March 31, 2004, the 
OEFC has had seven consecutive years of paying 
down its unfunded liability (often called the 
stranded debt), for a paydown from $20.55 bil- 
lion as at March 31, 2004, to $13,448 billion as 
at March 31, 2011— a reduction of about $7.1 bil- 
lion. Also, as provided for under the Act, the 
Ministry will be moving forward with proposed 
regulations under the Act on the determination 
of stranded debt and residual stranded debt. 

The determination of residual stranded 
debt from time to time is subject to estimation 
and forecast uncertainty because a residual 
stranded debt calculation includes forecasting 
what future dedicated revenues to the OEFC 
will be. Such revenues depend on the financial 
performance of OPG and Hydro One, as well as 
other factors such as assumptions about future 
tax and interest rates. 

This uncertainty has been reflected in the 
OEFC's 2011 annual report, which includes, as 
in previous years, an estimate of when the DRC is 
likely to end: sometime between 2015 and 2018. 
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Background 



Ontario's forests cover more than 700,000 square 
kilometres or about two-thirds of the province. 
More than 80% of the forests are on Crown land, 
and their management (that is, their harvesting, 
renewal, maintenance, and so on) is governed 
mainly by the Crown Forest Sustainability Act, 1994 
(CFSA). The CFSA is designed to provide for the 
long-term sustainability of Ontario's Crown forests 
and the management of Crown forests in such 
a way that they meet the social, economic, and 
environmental needs of present and future genera- 
tions. In addition, the Ministry of Natural Resources 
(MNR) has standing authority under Ontario's 
Environmental Assessment Act regarding recurring 
forest management activities on Crown land, sub- 
ject to conditions, which MNR must adhere to. 

Ontario's forest industry is an important source 
of employment in the province, especially in 
northern communities. In the 2008/09 fiscal year, 
employment within the industry was estimated at 
166,000 jobs. According to Statistics Canada, in the 
2009 calendar year, the value of Ontario's forestry- 
sector products (that is, the province's pulp and 
paper, sawmill, and engineered wood and value- 
added wood products) was estimated to be approxi- 
mately $12 billion. In recent years, the industry has 



experienced a significant decline due mainly to the 
increase in the value of the Canadian dollar and the 
recent economic downturn in the United States, 
which has affected demand for forest products made 
in Ontario. As a result, many mills in the province 
have closed, either permanently or temporarily, 
resulting in a reduction in timber harvest levels and 
associated forest management activities. 

As shown in Figure 1, most forest management 
activities on Crown land occur in an area of about 
365,000 square kilometres known as the Area of the 
Undertaking (AOU) . Forest management activities 



Figure 1: Area of the Undertaking (AOU) 

Source: Ministry of Natural Resources 
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are generally not approved in the land north of the 
AOU, where access is limited. Most of the land south 
of the AOU is privately owned. Productive forest 
within the AOU covers about 262,000 square kilo- 
metres; only about 190,000 square kilometres of this 
area are eligible for forest management activities, 
with the rest comprising provincial parks, private 
lands, and areas where forest management activities 
cannot reasonably take place due to the terrain. 

The Area of the Undertaking is divided into 41 
units, known as Forest Management Units (FMUs). 
Thirty-eight of the 41 FMUs are managed by forest 
management companies operating under a Sustain- 
able Forest Licence (SFL). Under an SFL, which 
may be granted for up to 20 years, the SFL holder 
is responsible for preparing a Forest Management 
Plan (FMP) and implementing the plan by build- 
ing access roads, harvesting trees, renewing/ 
maintaining the forest, monitoring its forest man- 
agement activities, and reporting the results of its 
monitoring to the province. The remaining three 
FMUs are managed by the Crown. Forest Resource 
Licences (FRLs), which allow an individual or com- 
pany to harvest in an FMU, are also granted by the 
province. Before an FRL can be issued, the individ- 
ual or company must come to an agreement with 
the holder of the SFL. The FRL holder will generally 
not be responsible for any forest renewal/mainten- 
ance activities subsequent to harvesting, because 
this responsibility typically remains with the SFL 
holder. The province has granted nearly 4,000 
FRLs, which have a maximum term of five years. 

In October 2009, the province realigned the 
responsibilities of the management of Crown 
forests between two ministries — MNR and the Min- 
istry of Northern Development, Mines and Forestry 
(MNDMF). MNR, for the most part, is responsible 
for the overall stewardship of Ontario's Crown 
forests. Its key responsibilities include approving 
FMPs prepared by SFL holders, overseeing the for- 
est renewal and other activities of these companies, 
and public reporting on the health and sustain- 
ability of the Crown forests. MNDMF is responsible 
mainly for the business and economic aspects of 



forestry. Its primary responsibilities include provid- 
ing the forest industry with access to Crown timber 
by granting SFLs, and the pricing, promotion, and 
marketing of Crown timber. 




The objective of our audit was to assess whether the 
Ministry of Natural Resources and the Ministry of 
Northern Development, Mines and Forestry (Min- 
istries) had adequate systems, policies, and proced- 
ures in place to ensure compliance with legislation, 
regulations, and policies and to reliably measure 
and report on their effectiveness in ensuring the 
long-term sustainability of Ontario's Crown forests. 

Senior management at both Ministries reviewed 
and agreed to our audit objective and associated 
audit criteria. 

Our audit included visits to the Ministries' head 
offices, to all three regional MNR offices, and to 
five of MNR's district offices (collectively, these 
district offices oversee 30% of the province's Crown 
forests), where we interviewed staff and reviewed 
pertinent files. We also visited MNR's Ontario For- 
est Research Institute in Sault Ste. Marie and its 
Centre for Northern Forest Ecosystem Research in 
Thunder Bay and met with researchers who are sup- 
porting the sustainable management of Ontario's 
forests. We met with the Ontario Forest Industries 
Association, the Wildlands League (a chapter of the 
Canadian Parks and Wilderness Society), and the 
Environmental Commissioner of Ontario to obtain 
their perspectives on forest management in Ontario. 
As well, we visited two sawmills, a paper mill, and 
a Crown forest managed by a licensee in northern 
Ontario to gain familiarity with their operations. To 
obtain a perspective on forest management practices 
in other provinces, we visited British Columbia and 
Alberta and met with representatives from their 
respective forest ministries. 

In recent years, the Internal Audit Services 
responsible for both Ministries had issued a number 
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of reports on various aspects of the Ministries' 
forestry program. We considered the relevant issues 
noted in these reports in determining the scope and 
extent of our audit. In addition, the Crown Forest 
Sustainability Act, 1994 requires that each managed 
Crown forest undergo an independent audit by a 
registered professional forester every five years. 
At the time of our fieldwork, such audits had been 
recently completed on 12 Crown forests. Where 
appropriate, we incorporated the results of these 
audits into our audit work. 



Summary 



Before the enactment of the Crown Forest Sustain- 
ability Act, 1994 (CFSA), the province was directly 
responsible for managing Ontario's Crown forests, 
including regeneration. Under the CFSA, licensed 
forest management companies became responsible 
for overall forest sustainability planning and for 
carrying out all key forest management activities, 
including harvesting and forest renewal, on behalf 
of the Crown. The province's role in ensuring the 
sustainability of Crown forests has increasingly 
become one of overseeing the activities of the 
private-sector forest management companies. Such 
oversight is vital given that forests take upwards of 
70 years to re-grow and these companies have little 
immediate financial incentive to carry out appropri- 
ate renewal activities. 

Overall, we concluded that improvements are 
needed if the Ministry of Natural Resources (MNR) 
and the Ministry of Northern Development, Mines 
and Forestry (MNDMF) are to have adequate assur- 
ance that the key objective of the CFSA — to provide 
for the long-term sustainability of Ontario's Crown 
forests — is being achieved. Specifically, we noted 
the following: 

• The province considers a one-hectare harvest 
block to have successfully regenerated if it is 
stocked with a minimum of 1,000 trees (that 
is, 40% of what the harvest block can accom- 



modate) . Harvest blocks are also held to a 
silviculture success standard, which is a meas- 
ure of whether the appropriate or preferred 
trees have grown back. In the 2008/09 fiscal 
year (the latest period for which information 
was available at the time of our audit), we 
noted that about a third of the licensed forest 
management companies had not reported the 
results of their forest management activities, 
and MNR had not followed up with these 
companies. The two-thirds that had reported 
indicated that although 93% of the total area 
that had been assessed by the companies had 
met the province's minimum 40% stocking 
standard, only 51% of the total area assessed 
had achieved silviculture success. 
MNR's 40% stocking standard has not 
changed since the 1970s. Several other juris- 
dictions in Canada hold the industry to higher 
standards. In fact, we noted that one MNR 
region, on its own initiative, held companies 
managing Crown forests in its jurisdiction to a 
higher stocking standard. 
Before planting, seeding, or even natural 
regeneration can take place, it is often neces- 
sary to prepare a site to allow for regeneration 
to take place under the best possible condi- 
tions, thereby increasing the likelihood of suc- 
cess. It is also often necessary to subsequently 
tend the site, usually by spraying to kill off 
competing vegetation, to further increase 
the likelihood of regeneration success. On 
average, between the 2004/05 and 2008/09 
fiscal years, only about a third of the area 
targeted for regeneration either naturally or 
by direct seeding or planting was prepared 
and/or subsequently tended. Moreover, the 
average decreased over that five-year period. 
In accordance with the CFSA, all Crown 
forests are subjected to an Independent For- 
est Audit (IFA) once every five years. Several 
of the more recent IFA reports completed in 
the 2008 and 2009 calendar years expressed 
concern about inadequate site preparation 
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or about non-existent or inadequate tending 
practices that were leading to reductions in 
growth, yield, and stand densities, as well as 
to an increase in the time required for stands 
to reach free-to-grow status (meaning that 
the trees are free of insects, diseases, and high 
levels of competing vegetation) . 
« We noted that Forest Management Plans 
had been completed in accordance with the 
requirements of the CFSA and reviewed and 
approved by MNR staff. However, MNR had 
not ensured that the most accurate and up- 
to-date information on forest composition, 
wildlife habitat, and the protection of these 
habitats was made available at the time the 
plans were prepared. 

• With respect to the province's monitoring of 
the forest industry, we noted the following: 

• MNR did not maintain a complete list of 
all active harvest blocks in its compliance 
system to ensure that all harvest blocks 
could be identified for possible inspection, 
and not all of MNR's district offices used a 
risk-based approach for selecting blocks for 
inspections. Where problems were noted, 
repeat offenders often did not receive 
appropriate remedies such as a penalty or a 
stop-work order. 

• The forest industry is required to report 
its renewal activities annually to MNR. To 
verify the accuracy of the reporting, MNR 
implemented a Silviculture Effectiveness 
Monitoring program. However, its district 
offices were not completing many of the 
required "core tasks" in the program. 
Where problems were noted, little follow- 
up action was being taken. 

• We noted that a good process was in place 
to select the team that conducted the 
Independent Forest Audits, but that defi- 
ciencies detected during such audits were 
not being addressed in some cases. 

• The average annual harvest in the last five 
years has been only about 63% of what was 



planned, and has decreased from almost 
80% of planned in the 2004/05 fiscal year to 
about 40% of planned in the 2008/09 fiscal 
year. The shortfall is usually due to existing 
licensees with sole rights to harvest Crown 
timber not having a market for the timber. 
There are indications that other companies 
that currently do not have access to timber in 
Ontario's Crown forests can market Ontario 
wood. A November 2009 competition for 
unused Crown wood initiated by MNDMF 
resulted in the allocation of approximately 
5.5 million cubic metres of timber that other- 
wise would not have been harvested. About 
25% of the winning proponents were new 
mills that plan to invest in the province as a 
result of this competition. At the time of our 
audit, MNDMF had no plans to hold similar 
competitions in the near future. In fact, we 
noted that MNDMF does not monitor on an 
ongoing basis whether there is excess supply 
of Crown wood that could be reallocated 
to others who might be able to market the 
timber. 

• Measures and controls did not fully ensure 
that Crown forest revenue was appropriately 
calculated and received on a timely basis and 
that trusts established to fund forest renewal 
expenditures incurred by forest management 
companies were adequately administered and 
funded. 

MNR could also enhance the usefulness of the 
information presented in its annual report on for- 
est management by comparing actual levels of key 
forest management activities — such as harvesting, 
regeneration (whether occurring naturally or 
assisted by planting or seeding), site preparation, 
and tending — to planned levels and providing 
explanations for significant variances. 
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Detailed Audit Observations 



OVERALL MINISTRIES' RESPONS 



MNR and MNDMF collectively aim to ensure 
that the management of Ontario's forests pro- 
vides healthy, sustainable forested ecosystems; 
enables a thriving and viable forest sector; and 
supports the livelihood of forest-dependent 
communities. The ministries work together in 
delivering on the requirements of a rigorous 
legislative and policy framework governing the 
management and use of Ontario's forests. This 
framework is regularly reviewed and assessed. 

Monitoring of approved operations and the 
results of management activities are essential 
components of Ontario's forest management 
framework. Monitoring is undertaken by forest 
companies, the ministries, and independent 
auditors to ensure that policies are effective 
and forest management objectives are being 
achieved. 

MNR and MNDMF recognize the need to 
consider new opportunities, and they have 
responded with the Ontario Forest Tenure Mod- 
ernization Act to modernize forest tenure and 
pricing, a provincial wood supply competitive 
process to ensure that the best use is made of 
our available forest resources, and the revision 
of forest management guides to incorporate the 
latest scientific information. 

MNR and MNDMF are committed to the 
continuous evaluation and improvement of the 
forest management program. The ministries 
appreciate the review undertaken by the Office 
of the Auditor General of Ontario and are com- 
mitted to responding to the recommendations 
to improve the forest management program in 
Ontario. 



SUSTAINABLE FOREST MANAGEMENT 

Before the enactment of the Crown Forest Sustain- 
abilityAct, 1994 (CFSA), the Ministry of Natural 
Resources (MNR) was responsible for the direct 
management of Ontario's Crown forests, includ- 
ing regeneration. Under the CFSA, licensed forest 
management companies, rather than MNR, became 
directly responsible for forest sustainability plan- 
ning and harvesting and are required to carry out 
forest renewal on behalf of the Crown. MNR's role 
in ensuring the long-term health of Crown forests 
has progressively become one of setting renewal 
standards and targets for forest management 
companies to meet, and overseeing the activities of 
these companies. 

It is critical for MNR to capably oversee the 
forest industry and ensure that the private-sector 
forest management companies are managing 
Crown forests in accordance with standards that 
ensure those forests' long-term health. Setting 
appropriate renewal standards and effective over- 
sight are all the more important in the case of forest 
renewal; with forests taking upwards of 70 years 
to re-grow, these companies have little immediate 
financial incentive to carry out appropriate renewal 
activities. 

Forest Renewal 

Regeneration Standards 
Under the CFSA, all areas harvested (excluding 
certain areas, such as roads) are required to be 
regenerated. Forests can regenerate naturally; they 
can also be regenerated by direct seeding or by 
planting trees. For each harvest block within a har- 
vest area, the Ministry has two key standards with 
respect to forest regeneration — overall regeneration 
success, and the success of silviculture, which is the 
practice of controlling the establishment, growth, 
composition, health, and quality of forests to meet 
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diverse needs and values. In a typical harvest 
area within the province, a one-hectare harvest 
block can accommodate approximately 2,500 
trees spaced about two metres apart. The province 
considers the harvest block to have successfully 
regenerated if it is stocked with a minimum of 
1,000 trees (40% of what the harvest block can 
accommodate) that have been declared free-to- 
grow (that is, the trees have good growth rates and 
are free from any insects, diseases, and high levels 
of competing vegetation) . Harvest blocks are also 
held to a silviculture success standard, which is a 
measure of whether the appropriate or preferred 
trees have grown back. 

Forest management companies are required to 
report regularly to MNR the results of the assess- 
ments completed on areas harvested seven to 10 
years previously within Forest Management Units 
(FMUs) and whether these areas have achieved the 
province's stocking and silviculture standards. In 
the 2008/09 fiscal year (the latest year for which 
information was available at the time of our audit), 
we noted that about a third of the forest manage- 
ment companies had not reported the results of 
their forest management activities in 2008/09, and 
MNR had not followed up with the companies that 
had not reported. The two-thirds that had reported 
indicated that although 93% of the total area that 
had been assessed by the companies had met the 
province's minimum 40% stocking standard, only 
51% of the total area assessed had achieved silvicul- 
ture success. 

The province's minimum 40% stocking standard 
has been in place since the 1970s. Several other 
provinces within Canada hold the forest industry in 
their respective jurisdictions to much higher stock- 
ing standards. In the 2009/10 fiscal year, one MNR 
region took the initiative of incorporating a higher 
minimum stocking standard in Forest Management 
Plans for FMUs within its jurisdiction and also 
began requiring that the trees be well dispersed — 
specifically, that at least 75% to 80% of the harvest 
block be covered. The province's current stocking 
standard does not require this. At the time of our 



audit, we were informed that MNR was in the 
process of reviewing the 40% stocking standard, 
because it felt that new science-based standards 
were needed in order to "raise the bar" and result in 
better renewal practices. 

Forest Regeneration 
Although assisted regeneration (direct seeding 
or planting) is a more expensive procedure than 
natural regeneration, on certain sites it is generally 
regarded as a more reliable option that yields a 
greater likelihood of establishing the desired spe- 
cies. A particular benefit of planting versus natural 
regeneration is that seedlings are germinated in 
greenhouses before planting, thereby providing a 
head start on regeneration. In addition, planting 
also allows greater control of stocking density. 
Before planting, seeding, or even natural regenera- 
tion can take place, it is often necessary to prepare 
a site to allow for regeneration to take place under 
the best possible conditions, thereby increasing 
the likelihood of success. Site preparation may 
consist of raking, tilling, or removing debris and 
undesirable competing vegetation. After planting or 
seeding, tending (that is, weeding and thinning) is 
usually required for some time to further increase 
the chance of silviculture success. 

On average, between the 2004/05 and 2008/09 
fiscal years, only about a third of the areas regener- 
ated either naturally or by direct seeding or plant- 
ing were site prepped and/or subsequently tended. 
During this period, there was a declining trend 
reported by forest management companies in the 
level of both site preparation (from 39% to 29%) 
and tending (from 45% to 35%) relative to natural 
and assisted regeneration. 

Forest regeneration activities prescribed in 
Forest Management Plans (FMPs) vary in inten- 
sity from inexpensive treatments such as natural 
regeneration following harvest to more costly treat- 
ments that involve site preparation, tree planting, 
vegetation management, and pre-commercial thin- 
ning. The FMPs prescribe preferred and alternative 
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treatments and give industry the option to carry 
out whatever treatments it considers appropriate. 
Forest management companies could, for the most 
part, avoid the more expensive and intensive treat- 
ments, opting for lower-end regeneration activities, 
and still be in compliance with their respective 
FMPs. However, continued use of lower-end practi- 
ces could yield poor regeneration results, especially 
with respect to establishing the desired species. 

In accordance with the CFSA, all FMUs are sub- 
jected to an Independent Forest Audit (IFA) every 
five years. The audit examines the effectiveness of 
both MNR and forest management companies in 
achieving the FMU's planned objectives and pro- 
vides an assessment of forest sustainability for that 
FMU. Several of the more recent IFA reports com- 
pleted in 2008 and 2009 expressed concerns with 
respect to inadequate site preparation or tending 
practices that lead to reductions in growth, yield, 
and stand densities and an increase in the time 
required for stands to reach free-to-grow status. 
For example, the most recent IFA report on a forest 
that has since reverted to the Crown reported the 
following: 

Underachievement of many planned 
renewal activities during the term was 
partially due to the reduced harvest level 
during the term but was largely due to 
selecting new less intensive treatments 
for many stands (for example, many areas 
planned for "Intensive" treatments — site 
preparation, planting and tending — 
were simply direct planted — a "Basic" 
treatment) . 



RECOMMENDATION 1 



To better ensure that the province's Crown 
forests are successfully regenerated after 
harvesting, the Ministry of Natural Resources 
(MNR) should: 

• follow up with those forest management 
companies that have not regularly reported 
on the results of their forest management 



activities in meeting the province's stocking 
and silviculture standards; and 
• conduct scientific studies and research into 
practices in other jurisdictions to ensure that 
the stocking standard is adequate to ensure 
that forest management companies are held 
to a regeneration standard that will success- 
fully renew harvested areas with the desired 
species. 

Where forest management companies opt for 
lower-end regeneration activities, MNR should, 
as part of its review of Forest Management 
Plans, ensure that there is adequate justification 
for these less-expensive treatments and assess 
whether the treatments will achieve planned 
renewal objectives. 



MNR RESPONSE 



MNR agrees that timely reporting is an essential 
component of monitoring the achievements of 
forest management companies in meeting the 
province's regeneration standards. Forest man- 
agement companies are required to report annu- 
ally to MNR on the results of any assessments 
that they have completed; however, companies 
are not required to conduct these assessments 
annually. Flexibility is provided in the system 
to allow a company to accumulate larger blocks 
that are assessed once every few years and 
in time for the preparation of the next Forest 
Management Plan (FMP) . Because of this, it 
is expected that not all companies will report 
each year. MNR will review its procedures for 
obtaining reports on the results of companies' 
regeneration assessments to ensure that where 
surveys are completed, the results are submit- 
ted on an annual basis. MNR will also follow 
up with any companies that have not regularly 
reported to ensure that they have a reasonable 
rationale for not doing so. 

Silviculture treatments that are necessary 
to successfully renew the forest and achieve the 
desired future forest condition are prescribed in 
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an FMP. MNR will review its approach for deter- 
mining minimum stocking levels set in FMPs 
and incorporate any necessary changes into the 
appropriate guidance documents. As part of its 
next update of the Forest Management Planning 
Manual, MNR will ensure that there is clear dir- 
ection for the provision of information to dem- 
onstrate that actual silviculture activities are 
consistent with the approved FMR Justification 
will be required where the level of less-intensive 
treatments deviates from planned levels in a 
given period. MNR will continue to monitor the 
effectiveness of natural regeneration and other 
less-expensive treatments as part of the ongoing 
Silviculture Effectiveness Monitoring program. 
The results of the provincial Silvicultural Effect- 
iveness Monitoring program will be analyzed to 
determine where improvements are needed in 
the existing silviculture framework. 



Forest Management Plans 

As noted earlier, the Crown Forest Sustainability Act, 
1994 requires that an approved Forest Management 
Plan (FMP) be in place for each FMU. FMPs are 
intended to safeguard the long-term sustainability 
of Crown forests and maintain biodiversity (that is, 
a variety of different plant and animal life). FMPs 
specify planned operations, including construction 
of access roads, levels of harvest and the associated 
levels of renewal, and maintenance over a 10-year 
term. Accurate and up-to-date information on for- 
est composition and wildlife habitat at the time the 
FMP is prepared is a key requirement for ensuring 
the sustainability of Crown forests. 

Forest Resource Inventory 
A forest resource inventory (FRI) provides informa- 
tion on, among other things, the composition, age, 
height, and stocking of individual species of trees 
within a forest. In our 1994 Annual Report, we com- 
mented that "an essential first step in any forest 



management process is a complete, accurate, and 
up-to-date forest inventory for each forest manage- 
ment unit." As well, in 2006, a task force imple- 
mented by the then-Minister of Natural Resources 
aimed at streamlining processes in the forestry sec- 
tor noted that a current FRI is needed to effectively 
carry out business, and that inaccurate values infor- 
mation leads to amendments and increased costs. 
MNRs standing approval under the Environmental 
Assessment Act for forest management activities also 
requires that an up-to-date FRI for each FMU be 
available for use in forest management planning. 
According to MNR, it takes three years to produce 
an FRI for an area, with the production predomin- 
antly entailing the taking of digital aerial imagery 
and the interpretation of that imagery using field 
data from surveys of a sample of plots. 

Realizing that its FRIs were becoming outdated, 
MNR allocated $7.5 million in the 2006/07 fiscal 
year and since then has allocated $10 million per 
year to enhance and update the province's FRIs 
using the latest technology. The current FRIs are, 
on average, 18 years old and therefore often do 
not contain accurate or complete information on 
the composition of forests within individual FMUs. 
FMUs are required to maintain a planning inventory 
for each FMP that contains updated forest descrip- 
tion information from forest management activities 
and natural changes to the forest. However, given 
that the basis of the planning inventories is the FRI, 
it is still essential for forest management planning 
that MNR has a complete and accurate FRI. MNR 
had initially intended to use the updated FRIs in 
forest management planning by 2010, but its cur- 
rent target is to have the new FRIs in place for forest 
management planning by 2014. 

Detection of Forest Resource Values 
Another requirement of the standing approval is 
that MNR maintain the most relevant and current 
information on such forest-related values as species- 
at-risk habitats, other species habitats, tourism 
values, and cultural and heritage values. To this 
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end, MNR maintains a values information system, 
which it is responsible for updating. Sustainable 
Forest Licence holders also have access to the sys- 
tem and use it to create value maps for use in FMPs 
and to adjust their operations according to any 
updated information. 

MNR's district offices receive funding every year 
to enable the collection of this data, but the amount 
of the funding provided is not consistent. We noted, 
for example, that funding to district offices in one 
region increased five-fold when FMPs within these 
districts came up for review. In other years, the 
funding was negligible. To facilitate the ongoing 
collection of data on values and hence enable more 
timely revisions to annual forest management oper- 
ations — especially with respect to wildlife habitats, 
which change continuously — the 2006 MNR task 
force recommended that "funding for values data 
collection projects, including entry of data into the 
corporate data repository, should generally be pro- 
vided to MNR Districts on an ongoing annual basis 
rather than be tied to the preparation of an FMP." 
MNR has yet to act on this recommendation. 

The Endangered Species Act, 2007 (ESA) lists 
many endangered and threatened species that need 
protection. MNR has determined that 42 of them 
are dependent on the province's Crown forests and 
are also likely to be affected by forest management 
operations. We noted that for approximately 15% 
of these species, no provincial prescriptions (that is, 
documents specifying the way the species should be 
protected — for example, by setting up buffer zones 
between the species and forest management oper- 
ations) had been developed at the time of our audit. 
Leaving development of such prescriptions up to 
the industry risks inconsistencies among FMUs. We 
also noted that one district had identified habitats 
for a number of the forest-dependent species-at-risk 
listed in the ESA in Crown forests in its jurisdiction, 
but these had not been entered in MNR's values 
information system. 



Update of Silviculture Guides 
MNR has produced silviculture guides on managing 
different species of trees in different regions. These 
guides are used by the forest industry when pre- 
paring FMPs. MNR's standing approval under the 
Environmental Assessment Act requires that MNR 
review these guides every five years to ensure that 
they reflect the most current scientific knowledge 
regarding the management of the different species 
of trees. MNR reviewed these guides in 2005 and 
concluded that, with only one exception, all of 
them required revision. At the time of our audit, 
MNR was still in the process of revising the guides. 



RECOMMENDATION 2 



In order that Forest Management Plans meet 
their objectives in ensuring the future sustain- 
ability of Crown forests, the Ministry of Natural 
Resources (MNR) should ensure that accurate 
and up-to-date information on forest compos- 
ition and wildlife habitat and the protection of 
these habitats is made available at the time the 
plans are prepared. MNR should also update any 
silviculture guides used in forest management 
planning on a more timely basis. 

| MNR RESPONSE 

MNR acknowledges the importance of using 
accurate and up-to-date information in the prep- 
aration of Forest Management Plans (FMPs) 
and makes significant ongoing investments in 
information and systems to support planning. 
In 2005, MNR assumed full responsibility for 
the production of the Forest Resources Inven- 
tory (FRI), and production to provide updated 
FRIs for use in the preparation of FMPs is on 
schedule. 

Values information, collected by MNR 
districts, is documented, and known values are 
verified during surveys that are conducted for 
purposes other than forest management (for 
example, moose aerial inventory surveys). The 
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collection of values information is continually 
augmented by district staff and industry part- 
ners during regular fieldwork. The planning 
system requires the immediate implementation 
of prescriptions to protect any newly identified 
values that may be affected by planned forest 
operations. 

Protecting species at risk and their habitats 
has always been an integral part of forest man- 
agement activities in Ontario. Specific provincial 
direction exists for 54 of the 65 species listed 
in the Endangered Species Act, 2007 (ESA) that 
are likely to be affected by forest management 
operations — or 83%. Direction for two further 
species is being developed. The development 
of additional habitat regulations and policy 
direction is ongoing as necessary to address out- 
standing species and any new species that may 
be listed under the ESA. 

MNR agrees that the standards and guide- 
lines in its forest management guides, including 
the silviculture guides, must be based on the 
most recent scientific understanding of sustain- 
able forest management practices. Following 
the 2005 review of the silviculture guides, work 
began immediately to address the three key 
items identified as requiring an update. Once 
that background research was completed, scop- 
ing of the new revised silviculture guide began 
in fall 2009. The silviculture guide revision 
project is now well under way, with completion 
anticipated in late 2013. 



Monitoring 

To assess compliance with approved FMPs and to 
evaluate progress with respect to forest regenera- 
tion, forest management operations carried out 
by the forest industry are subject to monitoring on 
three fronts: 

• compliance monitoring — that is, the inspec- 
tion of forest management operations to 



ensure that the operations conform to the 
approved plans or permits; 

• silviculture effectiveness monitoring, which 
is conducted to determine if forest renewal 
operations undertaken by the forest industry 
are yielding the desired outcomes; and 

• Independent Forest Audits (IFAs), which pro- 
vide an independent assessment of sustain- 
able forest management practices of FMUs. 

We noted the following with respect to the mon- 
itoring of forest management operations within the 
province. 

Inspection and Enforcement 
In Ontario, the forest industry and the Ministry of 
Natural Resources (MNR) are jointly responsible for 
ensuring that forest management operations (for 
example, construction of access roads, harvesting, 
and forest renewal/maintenance) comply with 
applicable legislation, regulations, and policies that 
are intended to ensure the sustainable management 
of Crown forests. In general, FMPs require Sustain- 
able Forest Licence holders to inspect all harvest 
blocks and report to MNR all suspected incidents 
of non-compliance within their FMU. MNR then 
confirms the suspected non-compliance (s) and 
determines the appropriate remedial action. 
MNR inspectors, in addition to verifying all non- 
compliance (s) reported by the industry, also carry 
out random and planned inspections. All inspec- 
tions conducted by MNR and the forest industry are 
documented in inspection reports. These reports 
are stored in a web-based system. Summaries of 
inspections are included in the provincial Annual 
Report on Forest Management. 

Inspections 

Although the forest industry is generally required 
to inspect all blocks harvested, MNR does not have 
adequate procedures for ensuring that the industry 
has carried out the required inspections. We noted 
that MNRs database did not contain a complete 
listing of all active harvest blocks; instead, it listed 
only those harvest blocks that had been inspected 



Forest Management Program 




by forest management companies. As a result, MNR 
cannot readily compare all active harvest blocks 
with those that have been inspected and follow 
up with companies regarding blocks that have not 
been inspected. 

Because the system provides for self-reporting 
by the industry, there is a risk that not all non- 
compliances will be reported. Between the 2005/06 
and 2009/10 fiscal years, MNR inspected an annual 
average of approximately 25% of the harvest 
blocks. We noted that during this period, the aver- 
age compliance rate on inspections carried out by 
the forest industry was approximately 96%, with 
the average compliance rate on MNR's inspections 
being about 87%. 

Although overall, MNR inspections yield a 
lower compliance rate, we noted that three of 
the five MNR districts we visited did not follow a 
risk-based approach in selecting which harvest 
blocks to inspect. The other two districts used risk 
assessment procedures to select harvest blocks for 
inspection. For example, one district had a good 
process that ranked blocks about to be harvested 
using criteria such as compliance history, public 
safety, and non-uniform boundary configurations; 
the district inspects 100% of the blocks deemed to 
be of high risk, 30% of blocks deemed to be of mod- 
erate risk, and only 10% of blocks deemed to be low 
risk. Adopting a similar risk-based approach in the 
selection of blocks for inspection across all regions 
would allow district offices to make the best use of 
limited resources. 

Enforcement 

When incidents of non-compliance (for example, 
access roads that are too wide, wasteful harvest 
practices, or operations taking place within a pro- 
tected area) are detected on inspections of harvest 
blocks by either the SFL holder or MNR, remedial 
actions available to MNR include written warnings; 
orders to stop, repair, or comply; administrative 
penalties; offence charges; and, as a last resort, 
suspension or cancellation of licences. 



On average, non-compliance problems were 
corrected nine months after the inspection date 
for inspections done between the 2005/06 and 
2009/10 fiscal years. When we reviewed a sample 
of non-compliance issues detected by the district 
offices we visited, we noted much longer delays 
in resolving these issues: in one case, 22 months 
after the inspection date. As of February 2011, 
there were 280 unresolved non-compliance issues 
province-wide, and these had been outstanding 
on average for nearly 23 months. Some of these 
non-compliance issues have remained unresolved 
because the forest management companies are no 
longer operating. MNR does not have a target for 
the time it should take to resolve non-compliance 
issues from the date of inspection. 

Our analysis also revealed that over the period 
from the 2004/05 fiscal year through the 2010/11 
fiscal year, there was a significant variance among 
MNR district offices (ranging from 0% to 80%) in 
the application of remedies when non-compliance 
issues had been detected. Although the industry 
does self-correct certain non-compliance issues, 
and remedies in these cases are not required, the 
significant variance indicates that in many situa- 
tions, remedial actions are not being consistently 
applied. In our testing, we noted that six forest 
management companies had more than 160 non- 
compliance issues collectively over the same six- 
year period, but no remedial action had been taken 
by the applicable district offices. 

We noted that repeat offenders often receive 
verbal or written warnings instead of remedies 
that might act as more of a deterrent — such as an 
administrative penalty or, where repeat violations 
are serious, cancellation of the licence. Administra- 
tive penalties account for fewer than 20% of the 
remedial actions applied by MNR over the five-year 
period from 2004/05 through 2008/09, and in 
only 5% of the cases have the companies been 
charged with an offence. Also, in the same five-year 
period, MNR cancelled only one Forest Resource 
Licence due to non-compliance. When we reviewed 
a sample of companies with repeat offences, we 
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noted that one company had 29 non-compliance 
issues (about a third of which involved waste- 
ful practices) over five years, but MNR applied 
the administrative penalty in only one of those 
instances. In another case, a licensee had 15 non- 
compliance issues, which included operating within 
a protected area and not following the annual work 
schedule submitted to MNR. Instead of administra- 
tive penalties, MNR gave verbal or written warnings 
in only four of these instances, and two of the 15 
non-compliance issues remained outstanding two 
years after being detected. 



RECOMMENDATION 3 



To improve its monitoring of forest management 
companies' operations for compliance with 
applicable legislation, regulations, and poli- 
cies, the Ministry of Natural Resources (MNR) 
should: 

• review its current compliance database to 
ensure that appropriate linkages are made 
to complete harvest block listings so that all 
harvest blocks can be identified for possible 
inspection; and 

• provide guidance to its district offices in 
adopting a risk-based approach for selecting 
blocks for inspection. 

MNR should also ensure that its district 
offices are more consistent and effective in the 
use of appropriate remedies to encourage com- 
pliance, especially for repeat offenders. 

| MNR RESPONSI 

MNR agrees with the recommendation and will 
review its current compliance system to ensure 
that appropriate linkages are made to the Forest 
Management Plan harvest block data so that 
all harvest blocks can be identified for possible 
inspection. 

MNR continues to evaluate and improve its 
program for monitoring forest management 
companies' operations for compliance. In 2010, 
MNR updated the Forest Compliance Handbook 



directive and procedures to provide clearer 
direction on the application of remedies. The 
program for monitoring forest management 
companies' operations for compliance is being 
considered as part of MNR's corporate review 
of natural-resource compliance monitoring. 
The corporate review process will lead to the 
development of a consistent ministry-wide 
approach to risk-based compliance monitoring. 
From this project, MNR will develop appropriate 
guidance on risk-based planning for considera- 
tion in the monitoring of forest operations. 



Silviculture Effectiveness Monitoring Program 
The forest industry is required to report its renewal 
activities annually to MNR. In 2006, to verify the 
accuracy of the reporting and to assess the effect- 
iveness of industry renewal activities in establishing 
new forests, MNR implemented the Silviculture 
Effectiveness Monitoring (SEM) program. The 
SEM program consists of a number of "core tasks" 
that MNR's district offices are to carry out to assess 
industry renewal efforts. These core tasks include 
surveying a sample of areas declared free-to-grow 
(that is, where trees have successfully regenerated 
and no further silviculture activities are required) 
by the industry, conducting field visits on a 
sample of sites where silvicultural activities have 
been reported to have been carried out by forest 
management companies, and mapping areas not 
yet considered free-to-grow. In the district offices 
we visited, we noted that the SEM program was 
not being carried out consistently. In the 2008/09 
and 2009/10 fiscal years, these districts on aver- 
age completed only 40% of the core tasks. In one 
region, some core tasks were not carried out at all. 
With respect to the delivery of the SEM program, 
we also noted the following: 

• One of the key core tasks is to conduct field 
surveys on 10% of the areas declared free-to- 
grow by the industry seven to 10 years after 
a site has been harvested. This independent 
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oversight is essential if MNR is to have at 
least some assurance that self -regulation 
by the industry with respect to successful 
regeneration is being accurately reported. In 
our sample, we noted that this core task had 
not been conducted in one FMU, and that the 
minimum 10% sample size was not met for 
over half the remaining FMUs. Another core 
task is to conduct field surveys on at least 5% 
of the areas declared free-to-grow, generally 
12 to 15 years after a site has been harvested. 
We noted that the districts we visited did not 
complete this task for 40% of the FMUs we 
sampled. In many cases, the districts that did 
perform the required survey did not meet the 
minimum 5% sample size. 

• The district offices generally took no follow- 
up action on areas found not to have met the 
free-to-grow standards. 

• Within the SEM program, there are no pre- 
scribed penalties that the districts can apply to 
encourage compliance. 



RECOMMENDATION 4 



To ensure that the Silviculture Effectiveness 
Monitoring (SEM) program adequately assesses 
the effectiveness of industry-reported renewal 
efforts in regenerating Crown forests, the dis- 
trict offices of the Ministry of Natural Resources 
(MNR) should complete all core tasks as out- 
lined in the program and follow up with forest 
management companies on sites found not to 
have met the free-to-grow criteria to ensure that 
the companies subsequently took appropriate 
remedial regeneration measures. 

To further enhance the effectiveness of the 
SEM program, MNR should consider prescrib- 
ing penalties that district offices can apply to 
encourage compliance. 



MNR RESPONSE 



MNR will take steps to improve the completion 
rate of the core tasks prescribed under the SEM 



program. MNR recognizes that determining the 
success rates of renewal efforts is a key compon- 
ent in an effective monitoring system. 

MNR will undertake a review of the SEM 
program and examine methods to allow earlier 
identification of sites that may not be regenerat- 
ing as originally planned. MNR will evaluate 
enhancements to the program to ensure that 
when remedial regeneration measures are 
required, appropriate incentives are in place to 
ensure that the measures are completed by the 
forest management company. 



Independent Forest Audits 
As noted earlier, every FMU in Ontario is subjected 
to an Independent Forest Audit (IFA) at least once 
every five years. The IFA is a requirement of the 
Crown Forest Sustainability Act, 1994 (CFSA), one 
of the conditions of MNR's standing approval under 
the Environmental Assessment Act, and a condition 
of all Sustainable Forest Licences (SFLs) . The IFAs 
purpose is to assess: 

• compliance with the CFSA; 

• compliance with the FMUs Forest Manage- 
ment Plan; 

• a comparison of planned versus actual forest 
management activities; 

• the effectiveness of forest management activ- 
ities in achieving planned objectives; and 

• where applicable, a licensee's compliance 
with the terms and conditions of its SFL. 

An SFL is generally granted for a term of 20 
years. Every five years, based on the IFAs results, 
the Ministry of Northern Development, Mines and 
Forestry (MNDMF) decides whether or not the 
licensee has complied with the terms and conditions 
of its SFL and, if serious difficulties are noted, can 
decide not to extend the SFLs term for another five 
years beyond the remaining term. 

The CFSA requires that the team conducting the 
IFA be independent of the licensee and of MNR, and 
specifies that at least one registered professional 
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forester must be a team member. All IFA auditors 
are selected by a committee that is independent of 
the government and that verifies the auditors' quali- 
fications. To further ensure the independence of 
the IFA process, the funds to pay for the IFAs come 
from the Forestry Futures Trust (discussed later in 
this report), to which all SFL holders contribute. We 
concluded that this was a sound process and should 
ensure that IFAs are conducted by auditors who are 
independent of the SFL holder and MNR. 

Since the inception of the IFAs in 1996, four SFL 
holders have been found to be in non-compliance 
with the terms of their licence or the CFSA. The 
non-compliance issues related to poor planning, 
areas that were insufficiently regenerated, and 
poor reporting of regeneration data. In none of 
these cases was the normal five-year extension 
to the existing term of the licence added, but the 
SFL holder was allowed to continue managing the 
Crown forest until the next IFA. 

Upon the completion of an IFA, MNR or MNDMF 
and the SFL holder must submit an action plan to 
address reported deficiencies within two months 
of receiving the final report and a status report two 
years after submitting the action plan. In general, 
the action plans and status reports address recom- 
mendations related to forest management planning, 
plan implementation, monitoring, and achievement 
of forest sustainability. We noted that a number 
of the action plans and status reports for IFAs 
conducted between the 2004/05 and the 2008/09 
fiscal years had not been completed by the forest 
management companies on a timely basis. Action 
plans were up to 16 months late, and status reports 
were up to 18 months late. At the time of our audit, 
some of these documents were still outstanding. 

Several of the IFA reports that we reviewed from 
the 2008 and 2009 calendar years also expressed 
concern regarding action items from previous 
IFAs: in some instances, the action items were 
either never undertaken or only partly completed, 
whereas in other instances, the action items did not 
fully address the IFAs original recommendation. 
MNR indicated that the delay in completing the 



action plans might have partially been due to the 
licence holder changing or to the overlapping work- 
load of SFL holders also having to prepare Forest 
Management Plans. 



RECOMMENDATION 5 



The Ministry of Natural Resources should 
ensure that action plans and status reports that 
address the recommendations of the Independ- 
ent Forest Audits are completed on a timely 
basis and ensure that it assesses the extent to 
which previous recommendations were satisfac- 
torily addressed. 

| MNR RESPONSE 

MNR acknowledges the need to improve the 
timeliness of action plan and status report 
submissions in response to IFAs. Additional 
measures will be taken to ensure timely action 
plans and status reports, recognizing that 
these reports can be difficult to produce on the 
timelines prescribed when there has been a 
change in ownership of the company or when 
a practical approach to address a particular 
recommendation is difficult to determine. The 
requirements, timelines, and process for assess- 
ing the status of previous recommendations 
are outlined in the Independent Forest Audit 
Process and Protocol, which is being formally 
reviewed in 2011. The results of the review will 
inform ongoing improvements to the IFA pro- 
cess, including the process in place to assess the 
extent to which previous recommendations by 
auditors have been addressed. 



Planned versus Actual Harvest 

As noted earlier, the Crown Forest Sustainability 
Act, 1994 (CFSA) requires that an approved For- 
est Management Plan (FMP) be in place for each 
Forest Management Unit (FMU). The FMPs, which 
are renewed every 10 years, must be certified by a 
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registered professional forester, who can work for 
the company managing the FMU. The FMP should 
set goals for forest sustainability and biodiversity. 
Based on the set goals, the FMP stipulates planned 
harvest and renewal levels aimed at ensuring the 
health of Crown forests so they can provide sus- 
tainable benefits (such as timber and commercial 
products, wildlife habitats, and recreational oppor- 
tunities) for the people of Ontario. 

As Figure 2 shows, between the 2004/05 and 
2008/09 fiscal years (the latest periods for which 
information was available at the time of our audit), 
actual harvest was significantly below planned 
harvest. During this period, the average annual 
planned harvest was about 290,000 hectares. How- 
ever, the average annual harvest in these five years 
was only about 180,000 hectares, or approximately 
63% of what was planned, with a decrease from 
almost 80% of planned in 2004/05 to about 40% of 
planned in 2008/09. 

Figure 3 shows that in the 2008/09 fiscal year, 
actual harvest was less than 50% of the planned 
harvest in about two-thirds of the FMUs. 

Under-harvesting over an extended period 
may have a negative impact on forest health and 
biodiversity. Ordinarily, natural disturbances, such 
as fires, insects, and windstorms, would do the job 
of regenerating forests. But where possible, the 
province suppresses such natural disturbances by, 

Figure 2: Planned vs. Actual Harvest, 2004/05- 
2008/09 (hectares) 

Source of data: Ministry of Natural Resources 
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for example, putting out forest fires and taking 
measures to control insect damage. The planned 
harvest and regeneration levels that are set in FMPs 
are intended to emulate the effects of these natural 
disturbances. Over the long term, the combination 
of continued under-harvesting and the suppression 
of natural disturbances can create an age-class 
imbalance — leaving only trees that are either very 
young or very old — in the province's forests. Older 
trees yield less timber, meaning that a much larger 
area needs to be harvested to produce the desired 
volume of timber. In a 2004 report titled Provincial 
Wood Supply Strategy, MNR noted that an age- 
class imbalance already existed in more than half 
the forests in the province, and that this situation 
would create a gap in the provincial wood supply in 
the future. 

In recent years, the competitiveness of the prov- 
ince's forest industry has declined, due partially to 
the increase in the value of the Canadian dollar. 
The economic troubles in the United States, along 
with an associated reduction in housing starts and 
in the consumption of goods and services, have also 
negatively affected Ontario's forest products indus- 
try in recent years. In those FMUs where licensees 
have sole rights to harvest Crown timber but do not 
have a market for that timber, the actual harvest 
tends to fall well short of the planned harvest. 

Figure 3: Actual Harvest vs. Planned, by Forest 
Management Unit (FMU), 2008/09 

Source of data: Ministry of Natural Resources 
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British Columbia government officials informed 
us that in 2003 the province took back approxi- 
mately 20% of its timber that was previously 
committed in long-term timber licences and now 
competitively reallocates the majority of this timber 
periodically to other market entrants. In addition, 
for timber that is still committed in long-term 
licences, the province takes back the unused por- 
tion of the annual allowable cut (which in Ontario 
is called the planned harvest) over a five-year per- 
iod and often competitively reallocates this timber 
to companies that are able to use the wood. 

There are indications that other companies that 
currently do not have access to timber in Ontario's 
Crown forests can market Ontario wood. In Janu- 
ary 2009, to attract new investment in the forest 
industry, the province initiated a staged competi- 
tion for Crown wood committed to long-term SFL 
holders that, according to FMPs, could have been 
harvested but was not being used by the licensees. 
The wood supply included merchantable wood 
(also called round-wood) and unmerchantable 
fibre (such as branches and the tops of trees) . The 
first stage of the competition was a Request for 
Expressions of Interest (RFEI), which was issued 
on January 20, 2009. In response to the RFEI, 
MNR received 131 submissions with proponents 
that were collectively interested in five times the 
wood supply that was considered available. Given 
this overwhelming interest, the province issued a 
Request for Proposals (RFP) for this unused Crown 
wood in November 2009. In response to the RFP, 
more than 100 proposals were received — many 
from new companies or mills that proposed to use 
the unused Crown wood to produce value-added 
products such as biofuel. At the time of our audit, 
nearly half these proposals had been approved, and 
the Ministry of Northern Development, Mines and 
Forestry (MNDMF) was in the process of putting in 
place agreements with the winning proponents for 
the utilization of approximately 5.5 million cubic 
metres of Crown timber that otherwise would not 
have been harvested. About 25% of the winning 
proponents were new mills that would be mak- 



ing investments as a result of their success in this 
competition. 

At the time of our audit, MNDMF had no plans 
to hold similar competitions in the near future, nor 
did MNDMF have a mechanism for monitoring, 
on an ongoing basis, any excess supply of Crown 
wood that could be reallocated. In 2010, MNDMF 
developed a province-wide wood supply and com- 
mitment database to identify excess supply that 
could be reallocated to existing and new process- 
ing facilities that have no access to Crown wood. 
However, MNDMF informed us that this database 
was only for use in the provincial wood supply 
competition discussed above and was not designed 
for ongoing use. Even though MNDMF requires that 
mills submit annual reports on wood utilization, 
this information is not used to update the data- 
base so that wood usage can be monitored on an 
ongoing basis to identify any excess supply. 

In June 2011, the government passed the 
Ontario Forest Tenure Modernization Act, 2011 
(OFTMA), which allows the province to estab- 
lish Local Forest Management Corporations 
(LFMCs) — Crown agencies that are governed by 
a predominantly local board of directors and that 
are responsible for managing Crown forests and 
overseeing the marketing and sale of timber in a 
given area. LFMCs would permit other companies 
access at a competitive price to Crown timber previ- 
ously committed in long-term licences. However, 
the legislation allows the piloting of only two such 
corporations within the next five years. The govern- 
ment also amended the CFSA to give the Minister 
of Northern Development, Mines and Forestry 
the authority to cancel existing SFLs. MNDMF 
informed us that it planned to establish the two 
pilot LFMCs by 2013 and that, in the meantime, it 
has no alternative under existing legislation other 
than to renew SFLs that come up for renewal even 
if it believes that the SFL holder will not harvest the 
allowable cut. 
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RECOMMENDATION 6 



To help ensure that forests are being managed on 
a sustainable basis and that harvest operations 
are carried out in accordance with approved 
plans, the Ministry of Northern Development, 
Mines and Forestry should: 
• enhance its ability to monitor on an ongoing 
basis the excess supply of Crown wood that 
can be reallocated to new companies that 
can use or market the wood; and 
conduct research into successful practices 
used in other jurisdictions to address signifi- 
cant variances between planned and actual 
harvests. 

NDMF RESPONSE 

Ensuring that planned harvest volumes are 
harvested and utilized for the benefit of Ontario 
is a key responsibility of MNDMF. As part of 
this responsibility, it is essential to determine 
whether unused wood supply allocations are the 
result of short-term market fluctuations or are 
systemic, requiring reallocations. MNDMF iden- 
tified significant volumes of unused wood sup- 
ply in the province and determined that wood 
supply allocations were no longer functioning 
as intended and therefore were no longer 
benefiting Ontario. A two-pronged strategy was 
developed to address this issue. To deal with 
immediate concerns, within the parameters of 
existing legislation and regulations, a provincial 
Crown Wood Supply Competitive Process was 
implemented with a goal to allocate unutilized 
wood to new and existing companies that would 
use the wood. In the longer term, MNDMF 
has undertaken an initiative to modernize its 
tenure and pricing system in an effort to allow 
better access to Ontario's wood supply, thereby 
improving the likelihood that planned harvest 
volumes will be used. 

The Tenure and Pricing Modernization team 
has conducted extensive research in areas such 



as economic models, practices in other jurisdic- 
tions, anti-wood-hoarding mechanisms, and 
conditions for competition. The Tenure and 
Pricing Modernization initiative is well under 
way and is expected to yield long-term benefits 
for the management of Ontario's forests. In the 
interim, while the results of the wood supply 
competition are being implemented and the 
longer-term Tenure Modernization outcomes 
are being determined, MNDMF continues to 
carry the responsibility to ensure that planned 
harvests are used. New tools have been 
developed and are currently in use to monitor 
wood supply use and to identify surpluses. 



CROWN FOREST REVENUE 

Stumpage Fees 

In accordance with the CFSA, the province receives 
direct payments from the forest industry in the 
form of a stumpage fee for every cubic metre of 
timber harvested. In the 2010/11 fiscal year, the 
province collected $94 million in stumpage fees. 
The fee has the following three charges: 

• A price charge that consists of two components: 

• a minimum charge per cubic metre of timber 
harvested, depending on the species, quality, 
and intended usage (for instance, pulp versus 
veneer) of the wood: this charge, which is 
adjusted annually, is designed to provide 

a minimum royalty to the province for the 
use of Crown wood; and 

• a residual value charge that varies depending 
on the market price of wood products: this 
charge is designed to provide an additional 
royalty to the province for the use of Crown 
wood. 

• Aforest renewal charge to provide funding 
for forest regeneration: this charge varies 
depending on the tree species and its antici- 
pated renewal cost. The vast majority of 
the forest renewal levy is held in the Forest 
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Renewal Trust and can be used only in regener- 
ating Ontario's Crown forests. SFL holders are 
reimbursed from these accounts as they carry 
out eligible silviculture activities on Crown 
forests. 

• A Forestry Futures Trust charge applied at $0.48 
per cubic metre of timber harvested: Funds 
accumulated in this trust mainly support the 
silviculture activities required to renew forests 
damaged by fire, disease, or insect infesta- 
tion. The trust also funds renewal activities 
where a licensee becomes insolvent, as well as 
expenditures related to IFAs. 

Figure 4 shows the allocation of the total stump- 
age fee into the different components, as well as 
disbursements out of the two trusts in the 2010/11 
fiscal year. 

Wood Measurement 

For the purpose of calculating the stumpage fee, 
nearly all Crown timber harvested is measured by 
the mills that receive the timber, and these mills 
provide information on the species of trees and the 
respective volumes received to MNDMF. Because 
stumpage fees are not collected on undersized or 
defective wood, MNDMF applies factors to the 
volume of timber reported by the mills to estimate 
the percentage of defective or undersized wood 
received. These factors are usually determined by 
checking the number of undersized logs in a sample 
of loads that mills have received. We noted that 
there was no overall provincial guidance on how 
these factors should be determined, and all three 
regions used different methods for determining the 
factors. For instance: 

• The minimum sample volume used by one 
region to determine the defect factor was 

50 cubic metres, whereas another region used 
1,000 cubic metres. 

• Each region had a different method of deter- 
mining the undersize factor: one region cal- 
culated the factor based on its own sampling, 
another combined its own sampling with the 



Figure 4: Allocation of Stumpage Fees and Trust 
Disbursements, 2010/11 ($ million) 

Source of data: Ministry of Northern Development, Mines and Forestry 
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mills' sampling, and the final region used only 
the mills' sample to determine the factor. 

• All three regions used different averages of 
sample data to determine the defect factor — 
from a three-year rolling average of sample 
data to an average of all sample data within 
the region's database. 

According to MNDMF guidelines, a scaling audit 
is to be performed on all mills every five to seven 
years. Such an audit verifies that the mills have 
adequate procedures for ensuring the accurate 
measurement of the Crown timber they receive. 
We noted that MNDMF has carried out an average 
of 10 such audits annually in the last nine years. 
At this rate, given that there are more than 200 
mills within the province that receive and measure 
Crown timber, MNDMF would take more than 20 
years to audit all mills — far longer than its internal 
guideline requires. 

Information System 
We analyzed the data between the 2005/06 and 
2010/11 fiscal years in the information system 
used by MNDMF to calculate stumpage fees and 
noted some examples of the system not having the 
necessary controls to ensure that stumpage fees are 
calculated correctly and that invoices are appropri- 
ately processed. For instance: 

• In our testing, we noted that a number of 
factors had been entered incorrectly within 
the system. We also noted that some factors 
within the system did not add up to 100%. For 
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example, a mixed-load factor estimated the 
volume of different species for only 87% of a 
given mixed load. As a result, MNDMF would 
not receive fees for 13% of a load if the mixed- 
load factor applied. 

• We noted that many factors had multiple 
effective and expiry dates, which could result 
in double billings. 

• More than 500 invoices were processed for 
species that forest management companies 
did not have a licence to harvest. We also 
noted that MNDMF processed 3,300 invoices 
totalling $5.4 million for species that the haul- 
ers were not authorized to haul. 

• Harvest approvals were granted to 16 com- 
panies that did not have a current Forest 
Resource Licence, yet the system allowed the 
entering of these approvals. 

Revenue Collection 

As of March 2011, $45 million of stumpage revenue 
was in arrears. On average, the amounts had been 
outstanding for approximately 19 months. About 
40% of the total amount outstanding related to 
companies that had declared bankruptcy, and 
another 35% related to companies that had worked 
out a repayment plan with MNDMF. We noted that 
about a third of the companies on the repayment 
plan had failed to meet their repayment obligations. 

The CFSA allows MNDMF to withhold licences 
or any approvals for harvest requested by the 
licensee if Crown charges are owed. In our sample, 
we saw no evidence that this was considered before 
approvals were granted to companies that were in 
arrears with respect to Crown charges. 



RECOMMENDATION 7 



To ensure that the province receives the proper 
amount of revenue for the use of Crown forest 
resources, the Ministry of Northern Develop- 
ment, Mines and Forestry (MNDMF) should: 



• develop overall provincial guidance for 
establishing wood measurement factors to 
ensure consistency and accuracy among the 
regions when determining stumpage fees; 

• increase the number of scaling audits per- 
formed each year to ensure that all mills 
are subject to the required audit every five 
to seven years in accordance with MNDMF 
guidelines; and 

• design and implement system controls in 
the stumpage fee information system so that 
invalid licence holders, and mills and haulers 
that are not authorized to receive and trans- 
port wood, are identified for appropriate 
follow-up. 

MNDMF should also formally assess the 
implications of renewing harvest licences where 
significant stumpage fees are outstanding. 

| MNDMF RESPONSE 

MNDMF will review existing regional sampling 
plans to ensure that they meet requirements, as 
well as evaluate consistency across regions and 
adjust standards where appropriate. MNDMF 
procedural guidelines dictate that all major 
companies will be audited once every five years 
and all other companies will be audited on a 
rotating basis. MNDMF will complete a review 
of the current procedure and assess whether 
revisions that add clarity and definition to the 
requirements are needed. As part of this review, 
MNDMF will consider the recommendation that 
the number of scaling audits performed each 
year be increased to a level where all mills are 
subject to audit every five to seven years. 

The current stumpage fee system provides 
information that would identify non-compliance 
issues relating to authorizations for the move- 
ment and measurement of Crown forest resour- 
ces. MNDMF has begun to design additional 
controls to the system to ensure timely identifi- 
cation and notification to allow for appropriate 
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follow-up action by both the Ministry of Natural 
Resources (MNR) and MNDMF. 

Collection of outstanding stumpage fees is 
a joint responsibility with MNR. When consid- 
ering the renewal of harvest licences where sig- 
nificant stumpage fees are outstanding, the two 
ministries work together to ensure that appro- 
priate measures are in place to collect outstand- 
ing stumpage fees before renewing the licence. 
There are many instances in which MNR and 
MNDMF have withheld licences and/or approv- 
als until companies have agreed to various types 
of repayment agreements, including repayment 
schedules and holdback agreements. The two 
ministries will review processes to determine if 
improvements are necessary. 



Forest Renewal and Forestry Futures Trusts 

As previously mentioned, under the CFSA, two 
trusts — the Forest Renewal Trust and the Forestry 
Futures Trust — have been established to fund forest 
renewal expenditures incurred by forest manage- 
ment companies. A portion of the stumpage fees 
that licensees pay for harvesting Crown timber goes 
toward funding these two trusts. In our audit, we 
noted the following with respect to the administra- 
tion and funding of the two trusts: 

• District managers determine the forest 
renewal levy for each FMU to fund the Forest 
Renewal Trust. Although MNR has issued 
guidance to help districts determine the 
renewal levy for different species of trees, we 
found that renewal rates across district offices 
for the same species of trees varied widely, 
from about 13% to 538% of the average, 
depending on the species. We acknowledge 
that differing factors — such as distance to 
harvest blocks, differing renewal objectives 
in FMPs, and the type of seedlings required — 
may result in some variance in the renewal 
charge, but we still questioned the magnitude 
of the variances among different districts. 



« SFLs require each licensee to maintain a min- 
imum balance in the Forest Renewal Trust net 
of expenses so that the trust can maintain a 
minimum overall balance of $95 million at the 
end of each fiscal year. The minimum balance 
is supposed to fund one year's forest renewal 
activities. However, we noted that this min- 
imum amount was last set in 1994. In 2009, 
an MNR working group concluded that the 
minimum balance should be based on an esti- 
mate of what the actual annual forest renewal 
obligation would be, rather than on an arbi- 
trary amount. At the time of our audit, MNR 
had not yet acted on this recommendation. 

• As of March 31, 2011, we noted that five SFL 
holders had not maintained their minimum 
balance totalling $4 million in the Forest 
Renewal Trust, contravening the terms of 
their SFLs. In 2008, a group of companies 
that had fallen far in arrears in maintaining 
their minimum balances declared bankruptcy, 
requiring MNR to obtain a Treasury Board 
approval for a $19 million top-up for the For- 
est Renewal and Forestry Futures trusts. 

• Before reimbursing any forest renewal 
expenses from the Forest Renewal Trust, 
MNR requires forest management companies 
to submit a list of invoices. MNR informed 
us that from the listings submitted, a sample 
of invoices over $1,000 is verified. For all 
expenses over $20,000, MNR procedures 
require that the expense be confirmed directly 
with the vendor. This is a good procedure, and 
our testing found that in 85% of our sample, 
MNR was able to provide evidence of this 
third-party verification. 

• In the past, the Forestry Futures Trust has not 
been able to fund some of the initiatives that 
it was intended to fund. For example, one 

of the purposes of the trust is to fund costs 
associated with pest control. In the 2006/07, 
2007/08, and 2009/10 fiscal years, aerial 
spray programs were conducted for jack 
pine budworm. But because the trust had 
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insufficient funds, part of the funding (about 
$13 million) had to come out of the province's 
consolidated revenue fund. One reason for the 
shortfall could be that the Forestry Futures 
Trust charge of $0.48 per cubic metre of tim- 
ber harvested, which funds the trust, has not 
changed since its inception in 1994. 
• MNR does not require SFL holders to provide 
any form of financial assurance that can be 
used to cover potential silviculture liabilities 
if a licensee becomes insolvent or surrenders 
its licence. One of the purposes of the Forestry 
Futures Trust as specified in legislation is 
to fund silviculture activities if a licensee 
becomes insolvent. However, as noted earlier, 
funding within the trust may not be sufficient 
to cover all potential silviculture liabilities. 
For example, in the case of one FMU where 
the SFL holder surrendered its licence, the 
province has been left with a significant silvi- 
culture liability that the trust may not be able 
to completely fund. In this regard, we noted 
that certain licensees in British Columbia are 
required to provide a security deposit of up 
to 100% of the expected silviculture cost of 
establishing a free-growing stand. 



RECOMMENDATION 8 



To ensure that the Forest Renewal Trust and the 
Forestry Futures Trust are sufficiently funded for 
their intended purposes, the Ministry of Natural 
Resources should: 

• review the significant variances in renewal 
rates calculated by district offices for the 
same species of trees to ensure that such 
variances are justified; 

• review the overall minimum balance that 
is to be maintained in the Forest Renewal 
Trust to ensure that the amount is a true 
reflection of the actual annual forest renewal 
obligation and ensure that licensees annu- 
ally maintain their portion of the minimum 
balance; 



• review the Forestry Futures Trust charge to 
ensure that it is sufficient to fund the initia- 
tives that the trust is intended to fund; and 
consider requiring SFL holders to provide 
some form of financial assurance that can be 
used to cover potential silviculture liabilities 
if a licensee becomes insolvent or surrenders 
its licence. 

MNR RESPONSE 



MNR recognizes that ensuring there are suf- 
ficient funds in Forest Renewal Trust accounts 
is critical to achieving the effective regeneration 
of Ontario's Crown forests. The Forestry Futures 
Trust account is also critical to ensure that 
Ontario can respond to the varied purposes of 
the trust, such as regenerating forests follow- 
ing natural disturbances, responding to forest 
pest outbreaks, and maintaining the province's 
forest resource inventory. It is also recognized 
that the trusts alone will not be adequate to deal 
with more catastrophic events, such as larger 
insect infestations, swaths of trees blown down 
by wind, and the occurrence of wildfires like 
the ones experienced in summer 2011. In these 
cases, nature is expected to run its course. MNR 
is currently working on improvements to the 
procedures for setting renewal rates. MNR will 
improve its process for analyzing the regional 
variances in renewal rates to determine if this 
variability is justified in terms of differences in 
local operating conditions and Forest Manage- 
ment Plan objectives. 

MNR has also begun developing a process 
for quantifying and maintaining a statement 
of outstanding silvicultural liabilities in order 
to evaluate the adequacy of the funds held in 
individual trust accounts. MNR monitors indi- 
vidual Forest Renewal Trust account balances 
monthly to ensure that there are sufficient funds 
in the accounts on March 31 of each year and 
has a process in place to collect a lump sum 
payment from those accounts that have not met 
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minimum balance requirements. The funding of 
silvicultural expenses resulting from insolvent 
licensees is currently addressed as one of the 
specified purposes of the Forestry Futures Trust. 
MNR is examining the funding model used to 
determine if it is adequate to meet the trust's 
purposes. 



REPORTING 

A requirement of MNR's standing approval under 
the Environmental Assessment Act is that MNR 
prepare an annual report on forest management 
and table the report in the provincial Legislature. 
Among other things, the standing approval requires 
that the annual report include information on the 
following key areas: 

• area and volume of Crown forest resources 
harvested; 

• government revenues from Crown charges; 

• the amount of regeneration, tending, and 
protection activities; and 

• silvicultural effectiveness. 

MNR informed us that it takes approximately 
18 months from the end of a given fiscal year to 
produce that year's report and that the time frame 
for tabling the report can vary. It compiles the 
information in the annual report from information 
it receives from the annual reports of individual 
FMUs. At the time of our audit, the most recent 
provincial annual report available was for the 
2008/09 fiscal year; it had been tabled in the Legis- 
lature in April 2011. MNR expected to complete the 
2009/10 annual report by October 2011 for subse- 
quent tabling in the Legislature. 

We reviewed the 2008/09 annual report and 
noted that overall, the information presented in 
the report met the requirements of the standing 
approval. However, the report presented only 
actual levels of forest management activities that 
took place in that fiscal year. We felt that the 
report's usefulness could be enhanced if it com- 



pared the actual levels of the key forest manage- 
ment activities — such as harvesting, regeneration 
(whether occurring naturally or assisted by plant- 
ing or seeding), site preparation, and tending — to 
planned levels and provided explanations for any 
significant variances. Annual reports on individual 
FMUs do contain comparisons of planned versus 
actual activities, and this information is publicly 
available. Nonetheless, it would be useful for MNR 
to summarize this information to facilitate province- 
wide comparisons. In this regard, we noted that 
both British Columbia and Alberta compare actual 
harvest levels with planned harvest levels in their 
public reporting. 



RECOMMENDATION 9 



To enhance the value of its annual report on 
forest management, the Ministry of Natural 
Resources should compare actual levels of key 
forest management activities — such as harvest 
and regeneration (that is, natural, planting, 
seeding, site preparation, and tending) — to 
planned or target levels and should provide 
explanations for any significant variances. 

| MNR RESPONSE 

MNR recognizes the need to continually 
improve the reporting on the management and 
status of Ontario's forests. MNR constantly 
evaluates approaches to forest reporting to look 
for efficiencies, enhance understanding, and 
improve access to information by the public, 
partners, stakeholders, and staff. MNR is adopt- 
ing a dynamic reporting cycle and instituting 
a more continual reporting of information 
through the Internet. Through these efforts, 
MNR will ensure that future annual reports on 
forest management will include an analysis on 
planned versus actual levels of key forest man- 
agement activities. 
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OTHER MATTER 

Licensing of Mills 

According to the Crown Forest Sustainability Act, 
1994 (CFSA), all mills that consume more than 
1,000 cubic metres of forest resources annually 
must have a forest resource processing facility 
licence. In March 2011, there were more than 200 
mills licensed in Ontario. 

To obtain a licence, mills are required by a regu- 
lation under the CFSA to submit a business plan to 
the Ministry of Northern Development, Mines and 
Forestry, which must be satisfied that the applicant 
has the ability to finance, operate, and manage the 
facility. Based on our testing, approximately 10% 
of the forest resource processing facility licences 
were issued to mills that had submitted business 
plans that did not demonstrate the applicant's abil- 
ity to adequately finance the facility. We also noted 
one mill had been operating since 2008 without a 
licence. 

Forest resource processing facilities are also 
required to submit an annual return that reports on 
the facility's operations. In our sample, two-thirds 
of the annual returns were either not submitted or 
not submitted on a timely basis. 



RECOMMENDATION 10 



The Ministry of Northern Development, Mines 
and Forestry should ensure that forest resource 
processing facility licences are granted only 



to those forest resource processing facilities 
that demonstrate that they have sufficient 
financial resources to operate, and ensure that 
forest resource processing facilities submit the 
required annual returns on a timely basis. 



The business plan requirement for licensing 
of mills in Ontario applies to a wide variety of 
facilities, from portable wood processors to full- 
scale pulp and paper mills. It also encompasses 
a variety of circumstances, such as the establish- 
ment of a new mill, expansion of an existing 
mill, or addition of a new product line. In all 
cases, the business plan submission must be 
adequate to satisfy MNDMF that the applicant 
has the ability to finance, operate, and manage 
the facility; in some circumstances, less informa- 
tion may be necessary to meet this threshold — 
for example, in the case of a long-established 
mill that is completing a minor expansion. 

MNDMF will review the consistency of its 
approach for ensuring facilities have demon- 
strated sufficient financial resources. In addi- 
tion, MNDMF will take measures to improve 
documentation of its assessment of sufficiency 
prior to issuing forest resource processing facil- 
ity licences. MNDMF will implement processes 
to ensure timely submissions of required annual 
returns from forest resource processing facility 
licence holders. 
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Ministry of Health and Long-Term Care 

Section 

3.06| Funding Alternatives for 
Family Physicians 




Background 



Ontario residents are eligible for provincially funded 
health coverage under the Ontario Health Insurance 
Plan (OHIP). The traditional method of compensat- 
ing primary-care physicians (also known as family 
physicians) for providing medical services has been 
to pay them a standard fee for each service per- 
formed, known as OHIP fee-for-service payments. 
The medical services covered and the standard fees 
payable are detailed in OHIP's Schedule of Benefits. 

Funding alternatives (known as alternate 
funding arrangements) for family physicians com- 
menced years ago, but over the last decade the 
Ministry of Health and Long-Term Care (Ministry) 
has significantly increased its use of these arrange- 
ments in order to, among other things, improve 
patient access to care and provide income stability 
for physicians. Under many of the arrangements, 
instead of receiving a fee for each service per- 
formed, physicians are paid an annual fee (called 
a capitation fee) to provide any of a specific list of 
services to each patient who agrees to see the phys- 
ician as his or her regular family physician. (Such 
patients are considered to have "enrolled with" the 
physician.) Services not covered by the capitation 
fee, including services provided to patients who are 
not enrolled, may generally be billed on a fee-for- 



service basis. By 2011, there were 17 types of alter- 
nate funding arrangements for family physicians, 
each with a different payment structure; 12 of these 
arrangement types were for physicians who treat a 
specialized population, such as maternity and pal- 
liative patients. 

Alternate funding arrangements are generally 
established and modified by the Physician Services 
Agreement between the Ministry and the Ontario 
Medical Association, which bargains on behalf of 
physicians in Ontario. This agreement — which 
has been negotiated every four years, starting in 
2000 — details the services that physicians are 
required to provide and the compensation that the 
province will pay for the services rendered. 

By the end of the 2009/10 fiscal year, more 
than 7,500 of the province's almost 12,000 family 
physicians were participating in alternate funding 
arrangements, and more than nine million Ontar- 
ians had enrolled with these physicians. Total fund- 
ing to all family physicians increased by 32%, from 
$2.8 billion to $3.7 billion, between the 2006/07 
and 2009/10 fiscal years. Of the $3.7 billion in total 
payments made to the province's family physicians 
in the 2009/10 fiscal year, more than $2.8 billion 
was paid to physicians participating in alternate 
funding arrangements, with $1.6 billion of this 
amount related to non-fee-for-service payments, 
such as annual capitation payments. 
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Audit Objective and Scope 




This year, our office performed two audits on 
funding alternatives (known as alternate funding 
arrangements) for physicians. The audit discussed 
in this section focused on the arrangements for 
family physicians, and the audit in Section 3.07 
focused on those for specialists. Our audit objective 
was to assess whether the Ministry has imple- 
mented systems and processes to monitor and 
assess whether alternate funding arrangements 
provide Ontarians with timely access to family 
physicians in a cost-effective manner. Ministry sen- 
ior management reviewed and agreed to our audit 
objective and associated audit criteria. 

Given the number of different alternate fund- 
ing arrangements available for family physicians, 
our audit focused primarily on the Family Health 
Group (FHG) and Family Health Organization 
(FHO) arrangements, and to a lesser extent on the 
Family Health Network (FHN) arrangement. In the 
2010/11 fiscal year, these three types of arrange- 
ments accounted for over 90% of family physicians 
participating in an alternate funding arrangement 
and over 90% of enrolled patients. 

Our audit work was conducted primarily at 
the Kingston and Toronto offices of the Ministry's 
Primary Health Care Branch. In conducting our 
audit, we reviewed relevant documents, analyzed 
information, interviewed appropriate ministry 
staff, and reviewed relevant research from Ontario 
and other jurisdictions. In addition, we employed 
a number of computer-assisted audit techniques 
to analyze patient-enrolment data, medical-claims 
data, and physician-registration records. As well, 
we reviewed and, where warranted, relied on the 
work completed by the Ministry's internal audit 
service team. 



The Ministry has made progress in its goal of 
increasing the number of Ontarians who have 
a family physician by encouraging physicians, 
through financial incentives, to switch from the 
traditional fee-for-service compensation model to 
alternate funding arrangements (mostly involv- 
ing multi-physician practices) . More than 90% of 
family physicians participating in these arrange- 
ments receive payments based on how many 
enrolled patients they have, as well as additional 
incentives and bonuses not available to physicians 
paid under the traditional fee-for-service model. 
Payments to family physicians through these 
arrangements more than doubled, from about 
$750 million in the 2006/07 fiscal year to over 
$1.6 billion in the 2009/10 fiscal year. During this 
time, the number of physicians participating in 
alternate funding arrangements increased by 11%, 
yet the number of patients enrolled in participating 
physicians' practices increased more substantially, 
by 24%, to more than nine million Ontarians. 

Although the Ministry intended alternate fund- 
ing arrangements to be more generous than the 
traditional fee-for-service model, the Ministry has 
not tracked the full cost of each alternate funding 
arrangement since the 2007/08 fiscal year. At that 
time, most family physicians participating in these 
arrangements were being paid at least 25% more 
than their counterparts compensated on a fee-for- 
service basis. In 2009/10, 66% of family physicians 
participated in an alternate funding arrangement, 
and these physicians received 76% of the total 
amount paid to all family physicians. Although the 
Ministry has some initiatives under way, it has not 
yet conducted any formal analysis of whether the 
expected benefits of these more costly alternate 
funding arrangements have materialized. 

The types of payments made under alternate 
funding arrangements are numerous and com- 
plicated, which has made it challenging for the 
Ministry to monitor physician compensation paid 
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through these arrangements or the extent to which 
the physicians have actually provided services 
required by their particular arrangements. The 
Ministry needs better information to determine 
whether the alternate funding arrangements are 
providing Ontarians with improved access to family 
physicians in a cost-effective manner and to be well 
prepared for the upcoming negotiations with the 
Ontario Medical Association in 2012. 

Some of our more significant observations 
include the following: 

• Along with alternate funding arrangements, 
the Ministry has established other initiatives 
to help people find a family physician. The 
Ministry estimated — on the basis of a survey 
it commissioned — that these initiatives have 
resulted in almost 500,000 more Ontarians 
having a family physician in 2010 than in 
2007. However, the survey also found that 
patients generally indicated that the wait 
times to see a physician had not changed sig- 
nificantly over the last few years. 

• Based on data from the 2007/08 fiscal year 
(the latest available at the time of our audit), 
family physicians paid through the Family 
Health Group (FHG) and the Family Health 
Organization (FHO) alternate funding 
arrangements earned on average $376,000 
to $407,000 (from which they pay overhead 
expenses), which was over 25% more than 
what, on average, family physicians were 
being paid under the traditional fee-for- 
service model. 

• The Ministry had adequate controls to ensure 
that no patient was enrolled with more than 
one family physician and was generally up to 
date on processing patient enrolments and de- 
enrolments with physicians. 

• Of the 8.6 million patients enrolled with 
either an FHO or an FHG, 1.9 million (22%) 
did not visit their physician's practice in the 
2009/10 fiscal year, yet the physicians in these 
practices received a total of $123 million just 
for having these patients enrolled. Further, 



almost half of these patients visited another 
physician, and OHIP also paid for those visits. 

• Although many more Ontarians are enrolled 
with multi-physician practices under the new 
alternate funding arrangements than in the 
2006/07 fiscal year, the wait time to see a 
family physician if they become sick has not 
changed as a result. Based on ministry survey 
results, while more than 40% of patients got 
in to see their physician within a day, the rest 
indicated that they had to wait up to a week or 
longer. 

• The annual capitation fee for each enrolled 
patient under an FHO arrangement can be 
40% higher per patient than the capitation 
fee for patients enrolled under a Family 
Health Network (FHN) arrangement, because 
almost twice as many services are covered 
under the FHO arrangements. Nevertheless, 
in the 2009/10 fiscal year, 27% of all services 
provided to FHO patients were not covered 
by the arrangement, and the Ministry paid 
an additional $72 million to the physicians 
for providing these services. Thirty percent 
of these services were for flu shots and Pap- 
smear technical services, yet the Ministry 
had not assessed whether it would be more 
cost-effective to have the annual capitation 
payment also include coverage for these and 
other relatively routine medical services. 

• Capitation rates in Ontario, similar to those in 
other Canadian provinces, are based only on 
the patient's age and sex, and do not consider 
the patient's health condition and health-care 
needs. As a result, the physician is paid the 
same for healthy patients (who require few 
or no medical services during the year) as for 
patients of the same age and sex who have 
multiple medical conditions. This situation 
can encourage physicians to de-enrol patients 
requiring more medical care, because a phys- 
ician can receive more funding for providing 
these patients with medical services under the 
traditional fee-for-service payment model. 
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| OVERALL MINISTRY RESPONSE 

The Ministry of Health and Long-Term Care 
(Ministry) welcomes the advice contained in 
this value-for-money audit. The audit acknow- 
ledges the progress achieved in increasing the 
number of Ontarians who have access to a 
family doctor. As the audit notes, large-scale 
changes to the traditional payment methods for 
family physicians were accompanied by a range 
of incentives and bonuses. 

Primary-health-care reform has been a 
significant government priority since the mid- 
1990s. In its reform initiatives, the Ministry 
has worked collaboratively with the Ontario 
Medical Association (OMA) to develop and 
promote alternate funding arrangements for 
family physicians that address patient-access 
issues. Since 1998, the Ministry, in co-operation 
with the OMA, has established new alternative 
funding models, and amended existing models, 
to promote family-physician participation and 
desired outcomes within the primary-care sec- 
tor. As of September 2011, the success of these 
models has been great: 7,739 Ontario doctors, in 
731 groups, are now providing primary health 
care to 9.6 million enrolled Ontario residents. 
Many of these groups are also participating in 
Ontario's Family Health Teams, and are now 
working with nurses, nurse practitioners, social 
workers, and others. Early evaluation results 
indicate that Ontario residents are pleased with 
these changes. However, as the Ministry moves 
forward in its primary-care reform initiatives, 
there is a need to balance the needs of the prov- 
ince's physicians with those of the patients, as 
well as the need to be accountable to taxpayers. 

Certain aspects of the agreements reflect 
early thinking on how incentives might encour- 
age participation by family physicians as well 
as enhance and improve preventive and com- 
prehensive primary health care. A thorough 
formal evaluation of these models will provide 
an opportunity to adjust the models based on 



experience and study. The stability provided 
by our success in attracting large numbers of 
physicians to these arrangements will also 
provide us with an opportunity to implement 
more complete and effective administrative and 
contract-monitoring mechanisms. 



Detailed Audit Observations 



OVERVIEW 

The Ministry's goals for alternate funding arrange- 
ments for family physicians include: 

• improving patient access to care; 

• promoting preventive care and chronic dis- 
ease management; 

• providing income stability for physicians; and 

• providing expenditure predictability for the 
government. 

To meet these goals, various alternate funding 
arrangements for family physicians have been nego- 
tiated between the Ministry and the Ontario Med- 
ical Association. Unlike traditional fee-for-service 
payments to physicians, these funding arrange- 
ments generally require physicians to provide at 
least some patient care outside of regular business 
hours, such as evening hours. As well, most of these 
funding arrangements require physicians to work 
in groups of three or more, to better ensure that a 
physician is available when a patient needs access 
to care. 

Physicians can choose whether or not to par- 
ticipate in an alternate funding arrangement, and 
also have the option of changing to a different 
alternate funding arrangement or going back to 
traditional fee-for-service payments. Therefore, to 
encourage physicians to join and remain in alternate 
funding arrangements, the Ministry has negoti- 
ated arrangements with different payment types. 
Selected payment types are shown in Figure 1. A 
few of the arrangements, such as the Family Health 
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Figure 1: Selected Types of Payments under Alternate Funding Arrangements for Family Physicians 

Prepared by the Office of the Auditor General of Ontario 


Type of Payment 


Description 


base capitation 


a fixed amount paid for each enrolled patient, based on age and sex, for providing services listed in the 
contract, regardless of the number of services performed or the number of times the patient visits the 
physician (for example, base capitation for FHOs ranges from about $58 to $521 per patient, and for 
FHNs from about $52 to $367) 


access bonus 


a portion of the base capitation that is reduced when enrolled patients seek care for services listed in the 
alternate funding arrangement from a physician outside the group the patients are enrolled with 


comprehensive 
care capitation 


a fixed amount paid for each enrolled patient, based on age and sex, for being responsible for a patient's 
overall care and co-ordinating medical services, such as referrals to other health-care providers 


complex capitation 


a fixed amount paid for enrolling a "hard-to-care-for" patient 


enhanced fee-for- 
service 


physicians bill OHIP and are paid at a rate higher than the traditional fee-for-service value for each patient 
service provided; the amount in excess of the traditional fee-for-service value is referred to as a "top-up" 
payment 


fee-for-service 


physicians bill OHIP and are paid the established fee per the OHIP fee schedule for each service provided 
to a patient 


incentives 


additional payments to physicians to provide specific services, such as patient care on weekends, 
preventive care, and diabetes management; encourage certain activities (e.g., enrolment of certain 
types of patients, such as hard-to-care-for patients); and compensate physicians for continuing medical 
education courses 


shadow billing 


physicians who receive base capitation funding can bill OHIP and be paid a percentage of the traditional 
fee-for-service amount for patient services listed in the alternate funding arrangement; physicians are 
generally eligible for either shadow billing or enhanced fee-for-service 


telephone health 
advisory service 


amount paid to physicians to be on call to provide after-hours telephone health advice for their enrolled 
patients 



Group arrangement, still pay physicians primarily 
on a fee-for-service basis, but at a rate higher than 
the traditional fee-for-service value; this approach 
is called enhanced fee-for-service. However, since 
such approaches provide neither physician income 
stability nor cost predictability, most of the funding 
arrangements, including the Family Health Organiz- 
ation and Family Health Network arrangements, pay 
physicians primarily through capitation. Under this 
approach, physicians receive a fixed annual amount 
(known as base capitation) for each enrolled patient 
(that is, each patient who signs a form to belong 
to the family physician's practice), based on the 
patient's age and sex, regardless of the number 
of times the patient visits his or her physician. 
This fixed annual amount pays for certain patient 
services, which are listed in each alternate funding 
arrangement contract (often called a "basket" of ser- 
vices). The listed services vary by funding arrange- 



ment — for example, Family Health Organization 
and Family Health Network arrangements have 
different listed services — so the capitation rates paid 
under each arrangement differ. 

Under the capitation-based arrangements, 
physicians are also allowed to bill OHIP for a por- 
tion, typically 10%, of the traditional fee-for-service 
value of the listed services whenever they actually 
provide each service to an enrolled patient. This 
approach, called shadow billing, provides the 
Ministry with information about the actual number 
of patients seen and clinical services provided. As 
well, physicians can bill the full traditional fee-for- 
service value for any services provided that are 
not listed in the contract (that is, not part of the 
"basket" of services), and for all services provided 
to non-enrolled patients. In addition, all alternate 
funding arrangements offer extra incentive pay- 
ments and bonuses designed to encourage certain 
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physician activities, such as providing diabetes 
management and preventive care, including screen- 
ing for breast and colon cancer. 

Family physicians who choose to participate in 
an alternate funding arrangement sign a contract 
with the Ministry, the Ontario Medical Association, 
and the other physicians, if any, participating in the 
arrangement. Under these agreements, physicians 
working in groups may either be signatory phys- 
icians (who are paid by the Ministry for meeting 
the agreement's obligations) or contract physicians 
(who are paid either primarily by the Ministry 
or by the signatory physicians, depending on the 
alternate funding arrangement) . Physicians are 
permitted to be a signatory in only one agreement 
and a contract physician in no more than three 
other agreements. As of March 31, 2011, about 40% 
of the groups used contract physicians, and about 
25% of the contract physicians worked for more 
than one physician group. 

Most family physicians participating in an 
alternate funding arrangement have chosen a 
Family Health Organization, Family Health Group 
or Family Health Network arrangement. These 
arrangements provide services to more than 90% 
of the enrolled patients, as shown in Figure 2. 
Physicians participating in these arrangements are 
compensated as shown in Figure 3. 

The Ministry introduced Family Health Teams 
in 2005 to bring together various interdisciplinary 
health-care providers, such as nurses, social work- 
ers, and psychologists, to work with physicians to, 
among other things, co-ordinate and enhance the 



quality of care for patients. While alternate funding 
arrangements pay for physician services, Family 
Health Team funding pays for other costs, such as 
the services of the interdisciplinary health-care pro- 
viders, as well as related administrative and other 
overhead costs. As well, one-time funding is pro- 
vided to Family Health Teams for office renovations 
and information technology. Physicians in certain 
alternate funding arrangements — including Family 
Health Organizations and Family Health Networks, 
but not Family Health Groups — may apply to the 
Ministry to establish a Family Health Team. How- 
ever, since traditionally physicians paid through 
fee-for-service are required to pay for most of their 
own overhead costs, including nursing and other 
staff costs, many more physician practices apply to 
the Ministry for funding than are approved. (For 
example, the Ministry received more than 70 appli- 
cations after its most recent announcement that 30 
new Family Health Teams would be approved.) As 
of March 2011, more than 2,200 physicians from 
almost 240 physician groups were participating in 
156 Family Health Teams, which received $244 mil- 
lion in Ministry funding in the 2010/11 fiscal year. 
The Ministry expected that an additional 21 teams 
it had approved would be operational by fall 2011. 

Alternate funding arrangements for family 
physicians are managed by the Ministry's Primary 
Health Care Branch. Other Ministry branches 
involved in administering the contracts include the 
Financial Management Branch, which is responsible 
for processing physician payments and conducting 
financial forecasting and reporting; the Health 



Figure 2: Physicians and Patients in Alternate Funding Arrangements, as of March 31, 2011 

Source of data: Ministry of Health and Long-Term Care 





# of Physician 




#of Enrolled 


% of Enrolled 


1 Alternate Funding Arrangement 


Groups/Practices 




Patients 


Patients 


Family Health Organization (FH0) 


352 


3,549 


4,877,000 


51 


Family Health Group (FHG) 


238 


3,056 


3,712,000 


39 


Family Health Network (FHN) 


36 


350 


356,000 


4 


other alternate funding arrangements 


73 


745 


584,000 


6 


Total 


699 


7,700 


9,529,000 


100 
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Data Branch, which is responsible for collecting and new physician registrations; and the Health 

statistics, analyzing trends, and calculating certain Solutions Delivery Branch, which is responsible for 

payments; the Registration and Claims Branch, developing information systems to support new 

which is responsible for, among other things, types of payments or changes in payment rates, 
processing medical claims, patient enrolments, 



Figure 3: Payment Methods for Selected Alternate Funding Arrangements for Family Physicians 

Prepared by the Office of the Auditor General of Ontario 



Type of Alternate Start 






Funding Arrangement Date 


How Physicians Are Paid 





Family Health 2006 Base and comprehensive care capitation, shadow billing, and incentives for enrolled 

Organization (FHO) patients 



Base capitation payment covers 118 listed services. Shadow billing is paid at 10% of the 

traditional fee-for-service value. 

Physicians also receive additional payments, including: 

• fee-for-service payments for any service not listed in the contract and for all services 
provided to non-enrolled patients 

• incentive payments for services such as preventive care, diabetes management, after- 
hours services, and enrolling unattached patients 

• complex capitation payments for "hard-to-care-for" patients 

• payments for being on call to provide after-hours telephone health advice to enrolled 
patients 

• $5,000 to $11,000 per year if they work in rural communities 

Funding of $12,500 to $25,000 per year is provided to practices with at least five 
physicians to hire an office administrator. 



Family Health Group 2003 


Enhanced fee-for-service and incentives for ministry-assigned patients and enrolled patients, 


(FHG) 


as well as comprehensive care capitation payments for enrolled patients 




Enhanced fee-for-service is 110% of the traditional fee-for-service amount. 




Physicians also receive additional payments, including: 




• complex capitation payments for "hard-to-care-for" patients 




• incentive payments for services such as preventive care, diabetes management, after- 




hours services, and enrolling unattached patients 




• payments for being on call to provide after-hours telephone health advice to enrolled 




patients 


Family Health Network 2001 


Base and comprehensive care capitation, shadow billing, and incentives for enrolled patients 


(FHN) 


Base capitation payment covers 56 listed services. The base capitation rate is lower than for 



Family Health Organizations, because fewer services are listed. Shadow billing is paid at 10% 
of the traditional fee-for-service value. 

As with Family Health Organizations, physicians also receive additional payments, including: 
• fee-for-service payments for any service not listed in the contract and for all sen/ices 



provided to non-enrolled patients 

• incentive payments for services such as preventive care, diabetes management, after- 
hours services, and enrolling unattached patients 

• complex capitation payments for "hard-to-care-for" patients 

• payments for being on call to provide after-hours telephone health advice to enrolled 
patients 

• $5,000 to $11,000 per year if they work in rural communities 

Funding of $12,500 to $25,000 per year is provided to practices with at least five physicians 
to hire an office administrator. 
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INITIATIVES TO IMPROVE ALTERNATE 
FUNDING ARRANGEMENTS 

Ministry initiatives to improve alternate funding 
arrangements for family physicians include negoti- 
ated changes to the Physician Services Agreement, 
as well as the following: 

• Since 2006, the Ministry has had a telephone 
survey conducted to obtain patients' perspec- 
tives on, among other things, access to care 
provided by family physicians. About 2,100 
Ontarians are surveyed every three months 
as part of this Primary Care Access Survey. 
The Ministry uses this information to develop 
strategies that will improve patients' access 
to care, such as helping patients find a family 
physician. 

• In 2007, the Ministry created the Quality Man- 
agement Collaborative (now part of Health 
Quality Ontario) to help Family Health Teams 
implement a team-based model of delivering 
primary health care. The organization's cur- 
rent objectives include using performance 
measurement to plan, test, and evaluate 
improvements in the organization and deliv- 
ery of primary health care. 

• In December 2008, the Ministry commis- 
sioned the Conference Board of Canada to 
conduct a five-year study on Family Health 
Teams to identify their successes and short- 
comings. Each year, the Ministry has been 
receiving interim study results, which focus 
on areas such as team functioning, patient 
access, and chronic disease management. 
The Ministry indicated that it will use the 
final report — expected in 2013 — to assist it in 
determining whether any changes should be 
made regarding Family Health Teams. 

« In February 2009, the Ministry launched 
Health Care Connect to help patients who 
have no family physician find one. By 
March 31, 2011, more than 100,000 patients 
had registered with the program, and 60% 
of those registered had been matched with a 



physician. The Ministry offers a bonus pay- 
ment to family physicians participating in 
alternate funding arrangements who accept 
patients who have been identified by the pro- 
gram as complex-vulnerable (that is, harder 
to care for). Almost 8,000 patients had been 
identified as complex-vulnerable, and almost 
6,000 of these had been matched with a phys- 
ician. This initiative is ongoing. 

ESTABLISHING ALTERNATE FUNDING 
ARRANGEMENTS 

Alternate funding arrangements for family phys- 
icians are agreements negotiated between the 
Ministry and the Ontario Medical Association, 
which bargains on behalf of physicians in Ontario. 
Since 2000, negotiations have taken place every 
four years, with any new funding arrangements 
or changes to existing arrangements requiring the 
agreement of both the Ministry and the Ontario 
Medical Association. Standard contracts were 
initially developed for each alternate funding 
arrangement, but negotiated changes are generally 
in other documents, such as the 2004 and 2008 
Physician Services Agreement. For example, the 
Physician Services Agreement contains information 
on new or additional fees (such as an increase to 
after-hours premiums and new fees for smoking 
cessation programs), as well as incentives and 
bonuses (such as bonuses for having patients par- 
ticipate in colorectal screening). 

We inquired whether the Ministry conducted an 
in-depth analysis of the anticipated costs of the new 
alternate funding arrangements before it entered 
into negotiations with the Ontario Medical Associa- 
tion, so that the total costs and expected benefits 
could be compared with those of the traditional 
fee-for-service model. Such an analysis would 
also be useful in ensuring that the Ministry had a 
well-informed bargaining position. Although the 
Ministry indicated that such an analysis had been 
performed, it was unable to locate this analysis. 
However, the Ministry did have information on the 
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expected costs of changes (such as new incentives) 
to alternate funding arrangements. 

Notwithstanding this, once the alternate fund- 
ing arrangements had been negotiated, we would 
have expected the Ministry to periodically analyze 
these arrangements to determine which features 
best met the Ministry's goals, including patient 
access to care, and how the relative cost of the 
new arrangements compared with the traditional 
fee-for-service model. This information would also 
be useful in future negotiations with the Ontario 
Medical Association. We were informed that the 
Ministry last performed a cost comparison of the 
alternate funding arrangements in the 2007/08 
fiscal year. At that time, the average gross income 
of family physicians paid solely on a fee-for- 
service basis was estimated to be about $285,000, 
whereas physicians participating in a Family Health 
Organization (FHO) made about $405,000, those 
in a Family Health Network (FHN) made about 
$360,000, and those in a Family Health Group 
(FHG) made about $375,000. From these amounts, 
physicians must pay the costs to run their practice, 
including any overhead costs — as well as, for FHOs 
and FHNs, amounts payable to participating con- 
tract physicians, if any. 

The Ontario Medical Association informs family 
physicians about alternate funding arrangements 
that they may join. Family physicians or groups of 
physicians interested in joining an arrangement 
then contact the Ministry. The Ministry verifies the 
credentials of applying physicians and ensures that 
each physician is a signatory physician under only 
one agreement and is a contract physician under no 
more than three agreements. 

Contracts, which set out the compensation 
arrangements as well as the services that are 
required to be provided, are signed by the signa- 
tory physicians who initially form the group and 
by representatives of the Ministry and the Ontario 
Medical Association. As well, all signatory and 
contract physicians participating in the group are 
required to sign a physician declaration form that, 
among other things, binds them to the terms of the 



contract. In the sample of contracts we tested for 
physicians participating in an alternate funding 
arrangement between April and December 2010, 
we found that 13% of physicians in FHGs and 18% 
of physicians in FHOs had not signed either the 
contract or the declaration form. There is a risk that 
physicians who have not signed the contract and/or 
the declaration form may not fully understand their 
obligations and, for example, might not provide the 
level of patient services required under the alter- 
nate funding arrangement. 



RECOMMENDATION 1 



To help ensure that alternate funding arrange- 
ments for family physicians meet the goals and 
objectives of the Ministry of Health and Long- 
Term Care (Ministry) in a cost-effective manner, 
the Ministry should: 

• periodically analyze the costs and benefits of 
existing alternate funding arrangements to 
determine whether the incremental costs of 
these arrangements are justified compared 
to the traditional fee-for-service model; 

• when negotiating alternate funding arrange- 
ments with the Ontario Medical Association, 
ensure that it has good information on the 
relative costs and benefits of new arrange- 
ments being considered as compared to the 
traditional fee-for-service compensation 
model, so that it is able to take a well- 
informed bargaining position; and 

• require all physicians to sign a contract 
before commencing participation in an alter- 
nate funding arrangement. 



| MINISTRY RESPONSE 

The Ministry supports this recommenda- 
tion, and as it moves forward to negotiate or 
renegotiate alternate funding arrangements, 
it will work toward full compliance with this 
recommendation. 

The Ministry agrees with the need to evalu- 
ate the costs and benefits of existing alternate 
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funding arrangements to ensure that the goals 
established under primary-health-care reform, 
particularly improved access to comprehensive 
primary-care services, are being achieved. The 
Ministry will commission a formal external 
evaluation of the two main alternate funding 
arrangements: Family Health Groups and 
Family Health Organizations. The results of 
the evaluation will inform amendments to the 
alternate funding arrangements to maximize 
benefits of the models and improve patient 
access to quality comprehensive primary-care 
services in Ontario. 

The Ministry will continue the practice that 
any new alternate funding arrangements, as 
well as any amendments to existing alternate 
funding arrangements, are fully costed prior 
to negotiations, and as negotiations proceed, 
with the Ontario Medical Association (OMA), 
to ensure that the Ministry is negotiating from 
a strong knowledge base. As was the practice 
in the negotiation of the 2008 Physician Ser- 
vices Agreement (PSA), the 2012 negotiations 
with the OMA will be based on an approved 
mandate with costed proposals and data to sup- 
port proposed changes to physician payments. 
Further, as with the 2008 PSA, expenditures for 
new initiatives under future agreements will 
be tracked and compared to projected costs to 
identify issues for review by the Physician Servi- 
ces Committee. 

The Ministry will implement procedures to 
ensure that payments under alternate funding 
arrangements do not commence until signed 
contracts are in place. 



ENROLLED PATIENTS 

By enrolling with a physician who is participating 
in an alternate funding arrangement, patients 
agree to seek treatment mainly from this physician 
or another physician working in the same family 



practice, except in an emergency. Patients are not 
required to enrol, and family physicians are not sup- 
posed to refuse medical care to any of their patients 
who prefer not to enrol in the new arrangement. 

In order to enrol with a family physician who is 
participating in an alternate funding arrangement, 
a patient must sign an enrolment form. Physicians 
forward the forms to the Ministry. The Ministry 
verifies that the patient has a valid Ontario health 
card and is therefore eligible for Ontario health 
insurance: if so, the Ministry records the patient's 
enrolment in its Client Agency Program Enrol- 
ment (CAPE) database, which lists all patients who 
have ever enrolled or de-enrolled with a family 
physician. If the patient was already enrolled with 
another physician, CAPE automatically changes the 
patient's enrolment to the new physician. Based on 
our analysis of the CAPE database, the Ministry's 
controls were adequate to ensure that no patient 
was enrolled with more than one physician as of 
April 2011. 

At the time of our audit, the Ministry was 
processing more than 100,000 enrolment requests 
and 10,000 de-enrolment requests every month. 
We also noted that the Ministry was up to date 
on entering and removing patients from the 
CAPE database. However, procedures need to be 
enhanced to help identify enrolled patients who 
may no longer be seeing the physician they are 
enrolled with, even though this physician is still 
being paid for being their family physician. For 
example: 

• There is no tracking of enrolled patients who 
rarely if ever visit the family physician they 
are enrolled with. We identified 1.9 mil- 
lion patients enrolled with either an FHO 
or FHG physician who had not visited their 
physician's practice at all in the 2009/10 
fiscal year. Almost half visited another 
physician, who received a fee-for-service 
payment for the medical services provided. 
The physicians with whom the patients were 
enrolled received a total of $123 million (after 
deducting the fees paid to the other physicians 
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visited) just for having these patients enrolled. 
The Ministry indicated that because capitation 
payments are based on the average level of 
physician services used by persons of the same 
age and sex, it expected payments for patients 
who seldom or never visited their physician to 
be offset by patients who require a high level 
of care. 

Although the Ministry identifies the total 
number of times in the month that a phys- 
ician's enrolled patients seek services from 
outside their physician's practice, the Ministry 
does not track this information by patient. Of 
the patients who had made at least one visit to 
the FHO or FHG they were enrolled with, our 
analysis identified 400,000 patients who saw 
a family physician outside of their physician's 
practice more often than they saw a family 
physician in the practice with which they were 
enrolled during the 2009/10 fiscal year. 
Alternate funding arrangements require that 
enrolled patients reside within 100 kilometres 
of the physician's practice. However, the 
Ministry has no procedures for identifying and 
de-enrolling patients who move outside their 
physician's practice area. For example, patient 
address changes in the Ministry's registered 
persons database, which lists all persons 
eligible for Ontario health insurance, do not 
trigger a check of enrolled patients. Even if 
they did, the registered persons database is 
not always updated on a timely basis. For 
example, if a patient moves outside the coun- 
try, the Ministry will be notified only if the 
patient fails to renew his or her photo health 
card (which could be up to five years after the 
move). Moreover, since 2.7 million enrolled 
patients still use the red-and-white health 
cards, which have no expiry date, it may be 
even more difficult to detect when they move. 
The Ministry indicated that patients are 
legally responsible for telling the Ministry if 
they no longer qualify for OHIP benefits — for 
example, when they move to another country. 



Patients who move are unlikely to see the 
physician they are enrolled with, and if they 
see a physician closer to their new home who 
does not enrol the patient as part of an alter- 
nate payment plan, their former physician will 
still be paid the annual capitation fee, and the 
new physician will be remunerated through 
the traditional fee-for-service model, unless 
the Ministry is made aware that the patient 
has changed physicians. The Ministry indi- 
cated that physicians are contractually respon- 
sible for notifying the Ministry if a patient 
moves; however, we believe that physicians 
often would not know when a patient moves, 
especially given the number of patients who 
seldom or never see the physician they are 
enrolled with. The Ministry further indicated 
that physicians would de-enrol a patient if 
that patient saw another family physician, but 
given that almost half the patients who never 
saw the physician they are enrolled with did 
visit another physician, we questioned this 
assumption. 
In April 2011, enrolled patients represented 
about 70% of the population in Ontario. However, 
we noted that many people didn't know that they 
were enrolled. In fact, the Primary Care Access Sur- 
veys conducted on the Ministry's behalf for the year 
ended September 30, 2010, indicated that only 32% 
of respondents believed they were enrolled with a 
family physician. Based on these results, we ques- 
tioned whether patients understand what it means 
when their family physicians ask them to complete 
an enrolment form. There is also a risk that enrol- 
ment forms could be submitted by physicians for 
patients who have not agreed to enrol. 

According to the enrolment form, either the 
patient or the physician can end the enrolment 
relationship. The patient can end his or her enrol- 
ment by notifying either the physician or the 
Ministry (through Service Ontario). Most de-enrol- 
ments are initiated by a physician, who must com- 
plete a form requesting that the Ministry de-enrol 
the patient. This form indicates only broad reasons 
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for the de-enrolment, such as "physician ended the 
patient enrolment." De-enrolment requests from 
physicians may occur because de-enrolling the 
patient would be more lucrative for the physician — 
that is, the physician would be paid more money 
for the patient through traditional fee-for-service 
payments (which might be the case, for instance, if 
the patient visits the physician frequently) . 

Enrolment Size 

The Family Health Network (FHN) and Family 
Health Organization (FHO) arrangements require 
each physician group to enrol a minimum number 
of patients. For groups with three to five physicians, 
the minimum enrolment size is 800 patients per 
physician. Individual physicians in a group are 
allowed to enrol fewer patients if other physicians 
in the same group enrol more (for example, for a 
three-physician practice to meet its minimum of 
2,400 patients, two physicians might enrol 100 
patients each and the third physician might enrol 
2,200 patients) . For all groups with more than five 
physicians, the minimum is 4,000 patients. For 
instance, in a 10-physician group, each physician 
would be required to enrol only 400 patients — a 
situation that could decrease access for people 
in the area who have no family physician. Family 
Health Groups (FHGs) have no minimum enrol- 
ment requirements. 

Our analysis of the enrolment data indicated 
that: 

• 93% of FHOs and FHNs with five or fewer 
physicians that had been in operation longer 
than one year had met the minimum require- 
ment by enrolling 800 patients per physician; 
and 

• the median number of enrolled patients per 
physician varied from about 1,025 to 1,400 
depending on the arrangement type, as shown 
in Figure 4. 

The actual number of patients these physicians 
see may be higher, because all participating phys- 



icians can also provide primary-care services to 
patients who are not enrolled. 

As is the case in many other jurisdictions, there 
is no limit on the number of patients an individual 
physician can enrol, and there are no guidelines on 
the optimal number of patients a family physician 
can reasonably expect to care for, whether the 
patients are enrolled with the physician or not. 
Although there is no maximum number, the FHO 
and FHN contracts state that if a physician group's 
average number of enrolled patients exceeds 2,400 
per physician, capitation payments are reduced by 
half for those patients above 2,400. Our analysis 
indicated that as of December 31, 2010, there were 
12 physician groups, with a total of 38 doctors, 
that had enrolled more than 2,400 patients per 
physician. 

Hard-to-care-for Patients 

Some patients require more frequent visits to their 
family physician because of ongoing health issues. 
These patients are generally considered hard to 
care for, although the Ministry does not have a 
specific definition to identify such patients. 

Under the alternate funding arrangements, 
although capitation rates generally get higher as 
patients get older, the rates are not adjusted based 
on a patient's health needs. In 2009, a report by the 
Institute for Clinical Evaluative Sciences found that 
in comparison to physicians paid through enhanced 
fee-for-service arrangements, such as FHGs, 
physicians participating in Family Health Teams 
enrolled fewer sick patients and fewer patients who 

Figure 4: Median Number of Enrolled Patients per 
Physician, as of December 31, 2010 

Source of data: Ministry of Health and Long-Term Care 
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made frequent visits to their physician. The reason 
for this difference was not clear. 

To encourage physicians to enrol hard-to- 
care-for patients, the Ministry offers short-term 
incentives. In order for physicians to earn these 
incentives, a person who does not have a family 
physician must first call the Ministry's Health 
Care Connect service, which helps people find a 
family physician. Health Care Connect evaluates 
the level of medical services the person requires to 
determine if he or she is a "complex- vulnerable" 
person (that is, hard to care for). If the person is so 
designated, the physician who enrols this patient 
will receive a one-time signing bonus of $350. In 
addition, during the first 12 months of enrolment, 
an FHO or FHN physician will receive an extra $500 
in total capitation payments for the patient, and an 
FHG physician will receive 150% of the value of any 
fee-for-service claim submitted for the patient. 

After the first year, no additional bonuses or 
incentives are available to physicians caring for 
such patients. In the 2009/10 fiscal year, Health 
Care Connect co-ordinated the enrolment of 1,600 
complex-vulnerable persons. At the time of our 
audit fieldwork, the Ministry had not monitored 
whether any of these patients are de-enrolled 
by their physicians once the short-term financial 
incentives end. However, the Ministry indicated 
that Health Care Connect had not notified the 
Ministry of any problems with previously matched 
complex-vulnerable patients seeking its assistance 
again after one year. 



RECOMMENDATION 2 



To better ensure that alternate funding arrange- 
ments are cost-effective and that patients have 
access to family physicians when needed, the 
Ministry of Health and Long-Term Care should: 
• periodically review the number of patients 
who do not see the physician they are 
enrolled with, and assess whether continu- 
ing to pay physicians the full annual capita- 
tion fee for these patients is reasonable; 



• review the impact of its policy that allows 
practices with more than five physicians to 
enrol only 4,000 patients in total, rather 
than the 800 patients per physician required 
by practices with fewer physicians, to deter- 
mine the impact this policy has on access for 
people with no family physician; and 

• review the number of patients being de- 
enrolled by their physician to determine 
whether a significant number of these 
patients are in the hard-to-care-for category, 
and, if so, whether the current financial 
incentive arrangements should be revised. 

| MINISTRY RESPONSE 

The Ministry is supportive of conducting one 
or more policy reviews to evaluate whether 
the current enrolment-related provisions in 
the alternate funding arrangements contribute 
toward improved access to primary-care services 
for enrolled patients. The Ministry notes that 
significant research on the delivery of primary- 
care services is available to it, which can inform 
and support its policy reviews and correspond- 
ing contract amendments where necessary. 

The Ministry's policy reviews will address 
the issues identified in the audit report: 

• the appropriateness of paying capitation 
payments in respect of enrolled patients who 
do not access care from the physician they 
are enrolled with during a one-year period; 

• the impact on access to care resulting from 
controls on minimum enrolment size; and 

• the linkage between de-enrolment and 
patient complexity and whether payment 
incentives are required to ensure continued 
access to care. 

Work that is currently in progress by a joint 
Ministry and Ontario Medical Association 
working group, with support from the Institute 
for Clinical Evaluative Sciences, may resolve 
issues related to maintaining complex patients 
in capitation-based funding models. This group 
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is evaluating options for modifying the current 
age/sex capitation rate to include an acuity/ 
complexity modifier, and is expected to submit 
its final report in December 2011. 



PATIENT ACCESS TO PRIMARY-CARE 
SERVICES 

Hours of Services 

Contracts signed by physicians participating in 
alternate funding arrangements generally state 
that the physician group "shall ensure a sufficient 
number of physicians are available to provide the 
services during reasonable and regular office hours 
from Monday through Friday sufficient and con- 
venient to serve enrolled patients." The contracts 
do not further specify what constitutes "a sufficient 
number of physicians," "reasonable and regular 
office hours," or "sufficient and convenient to serve 
enrolled patients." According to the Ministry, these 
terms were intentionally not further defined for 
various reasons, including giving physicians flex- 
ibility in operating their practices and avoiding 
contract restrictions that would prevent physicians 
from joining alternate funding arrangements. 

Further, physician practices are required to 
provide at least one three-hour block of after-hours 
services per week for each physician in the group, 
to a maximum of five three-hour blocks per week 
for practices with five or more physicians. The con- 
tracts define "after-hours" as Monday to Thursday 
after 5 p.m. or any time on the weekend — that is, 
any time from Friday through Sunday. (Physicians 
are required to have some regular office hours on 
Fridays, but can also provide after-hours services 
any time outside of their regular hours that day.) 
Although the after-hours blocks must occur on dif- 
ferent days, unlike regular office hours, there is no 
requirement in the contract for after-hours services 
to be "sufficient" or "convenient." For example: 

• Where a practice has more than five phys- 
icians, the minimum number of after-hours 



services required per week is five three-hour 
blocks. Eighty-seven percent of Family Health 
Networks (FHNs), 64% of Family Health 
Organizations (FHOs), and 53% of Family 
Health Groups (FHGs) have more than five 
physicians in a group. Such FHNs, FHOs, and 
FHGs average from 10,000 to 24,000 patients. 
However, only one physician is required to be 
available during each after-hours block, and 
therefore we believe it would be worthwhile for 
the Ministry to assess whether these practices 
are providing sufficient evening and weekend 
availability to meet their patients' needs. 
• Even though some groups operate out of mul- 
tiple locations, the after-hours services need 
only be offered at one location, which may 
not be convenient for many of the enrolled 
patients. 

At the time they are established, physician 
groups participating in an FHN, FHO, or FHG 
provide their hours of operation to the Ministry. 
Only FHGs are contractually required to update 
the Ministry if their office hours change. Although 
the Ministry was not periodically monitoring 
changes to FHN, FHO, or FHG office hours at the 
time of our audit fieldwork, the Ministry indicated 
that this information could be obtained from the 
provider of its Telephone Health Advisory Service. 
The Ministry initiated a project in summer 2011 to 
collect information on the regular and after-hours 
schedules of FHNs, FHOs, and FHGs. 

We reviewed ministry information for the 
2009/10 fiscal year on which day of the week 
physician services were provided and on whether 
the physicians participating in alternate funding 
arrangements billed the services as having taken 
place during regular hours or after hours (for the 
latter, they receive a premium payment from the 
Ministry). For FHGs, FHOs, and FHNs, our analysis 
showed that less than 15% of patient services were 
provided on Fridays, and only about 6% of services 
were provided on Saturdays and Sundays. We also 
found that 92% of services were provided during 
regular office hours. 
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At the time of our audit, more than 100 FHG, 
FHN, and FHO groups were exempt from providing 
after-hours services. Exemptions can be obtained if 
more than 50% of physicians in the group provide 
certain other services outside regular hours (for 
example, hospital emergency-room coverage or 
obstetrical services) and the group obtains ministry 
approval. Physicians are not required to state how 
many hours of these services they provide or other- 
wise supply any proof that they are providing these 
services. Until 2011, exemption approvals had no 
expiry date. Starting in 2011, physician groups will 
be required to apply for the after-hours exemption 
annually, and every time a physician joins or leaves 
the group. The Ministry indicated that it reviewed 
physician groups' eligibility for exemptions a few 
years ago, and found that all exempted groups were 
eligible. However, no documentation of that review 
was maintained, or on how eligibility was confirmed. 

In February 2011, the Ministry conducted an 
ad-hoc review of "after-hours" claims submitted 
by FHNs, FHOs, and FHGs for June 2010 to deter- 
mine whether physician groups complied with the 
after-hours service requirements. Ministry results 
indicated that only 41% of FHNs, 60% of FHOs, and 
74% of FHGs were providing after-hours services in 
accordance with their contracts. Physician groups 
providing less than 40% of the required after-hours 
services were sent a letter requesting an explana- 
tion. Some groups said that they met the exemption 
criteria but had not known they required ministry 
approval, and other groups said that they provided 
the services but didn't bill the after-hours code, 
despite the premium payment they would receive 
for providing after-hours services. The Ministry 
informed us that it is now requesting explanations 
from the rest of the physician groups with less than 
100% compliance. 

Physicians' Service Levels 

The Ministry obtained some information on the 
wait time to see a family physician through its 
Primary Care Access Survey. Approximately 2,100 



Ontarians aged 16 and older were surveyed every 
three months. The most recent survey responses 
available for our review were for the year ending 
September 30, 2010. Where appropriate, we com- 
pared these survey responses with responses from 
previous years. Based on survey responses, the Min- 
istry projected that the number of Ontarians with a 
family physician increased by almost 500,000 from 
2007 to 2010. 

Overall, for persons with a family physician, 
responses to questions regarding access were 
similar, regardless of whether the person was 
enrolled with a physician in an alternate funding 
arrangement or the person's physician was paid 
under the traditional fee-for-service model. As well, 
the length of time patients currently waited to see 
a physician generally had not changed significantly 
from responses received approximately three years 
earlier, even though many more patients were now 
enrolled with a physician participating in an alter- 
nate funding arrangement practice. For instance: 

• For persons who needed to see a physician 
because they were sick, 27% of respondents 
with a family physician (whether enrolled 
with the physician or not) said they saw a 
physician the same day; an additional 17% 
saw a physician the next day, and 44% were 
able to see a physician within two to seven 
days. The rest waited longer. 

• For persons who needed to see a physician 
for monitoring an ongoing health problem, 
12% of respondents with a family physician 
(whether enrolled or not) said they saw a 
physician the same day, 10% saw a physician 
the next day, and 45% were able to see a 
physician within two to seven days. The rest 
waited longer. 

• Among respondents with a family physician 
(whether enrolled or not) who visited their 
physician, 93% were satisfied with the care 
they received. Respondents without a family 
physician were about 10% less likely to be 
satisfied with the care they received. 
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• Among respondents with a family physician 
(whether enrolled or not) who went to an 
emergency department, 15% went because 
their family physician was not available. The 
survey did not distinguish between regular 
office hours, weeknights, or weekends. 

• Among respondents with a family physician 
(whether enrolled or not) who went to a 
walk-in clinic, 47% went because their family 
physician was not available. Another 36% said 
they went to a walk-in clinic because it was 
easier or more convenient. Compared to the 
2008/09 fiscal year, there was a 10% increase 
in the number of respondents who used a 
walk-in clinic because their family physician 
was not available. 

We also noted that interim results of the Min- 
istry-commissioned study on Family Health Teams 
have indicated that enrolled patients were generally 
satisfied with their access to health services. 



RECOMMENDATION 3 



To ensure that alternate funding arrangements 
are meeting their goal of improving access to 
family physicians, the Ministry of Health and 
Long-Term Care (Ministry) should: 

• periodically monitor whether physicians 
participating in alternate funding arrange- 
ments provide patients with sufficient and 
convenient hours of availability, including 
after-hours availability, as required by the 
arrangements; and 

• conduct a formal review of whether alternate 
funding arrangements are meeting the goal 
of improving access, especially given that the 
Ministry's Primary Care Access Survey indi- 
cates little change in the last three years in 
the wait times for seeing a family physician. 

MINISTRY RESPONSE 

The Ministry supports the monitoring of 
alternate funding arrangements to ensure that 
patients have access to sufficient and convenient 



hours of availability, including after-hours 
availability. The Ministry will develop service 
standards and performance measures that can 
be used to ensure that physician practices are 
open and available to their enrolled patients. 

Performance measures will be incorpor- 
ated into the alternate funding arrangements 
to ensure that physicians participating in the 
arrangements are aware of expectations and 
mechanisms that will be used by the Ministry to 
monitor compliance. 



PAYING FAMILY PHYSICIANS 

Non-fee-for-service payments made under alter- 
nate funding arrangements for family physicians 
increased significantly between the 2006/07 and 
2009/10 fiscal years, with payments under the 
Family Health Organization arrangement (includ- 
ing $153 million relating to Family Health Team 
funding) accounting for most of the increase, as 
shown in Figure 5. During the same time period, 
the total number of physicians participating in 
alternate funding arrangements increased by about 
730 (11%) and the number of enrolled patients 
increased by almost 1.8 million (24%). Total fund- 
ing to all family physicians increased by 32%, from 
$2.8 billion to $3.7 billion, between the 2006/07 
and 2009/10 fiscal years. In the 2009/10 fiscal 
year, 66% of the total number of family physicians 
in Ontario participated in an alternate funding 
arrangement. They received 76% of the total 
amount paid to all family physicians. 

At our request, the Ministry determined that 
family physicians participating in an alternate fund- 
ing arrangement also earned $1.2 billion in addi- 
tional fee-for-service payments in the 2009/10 fiscal 
year for providing medical services not included in 
the list of services covered by the annual base capi- 
tation payment, as well as for providing services to 
non-enrolled patients. The Ministry does not know 
how much each physician who is participating in 
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Figure 5: Expenditures by Alternate Funding Arrangement, 2006/07-2009/10 ($ million) 

Source of data: Ministry of Health and Long-Term Care 



Funding Model 2006/07 
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Family Health Organizations (FHO) 1 


59 
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Family Health Groups (FHG) 2 


229 


269 


278 


277 


Family Health Networks (FHN) 1 


209 


276 


201 
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other 


256 


152 
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221 


Total 


753 


968 


1,311 


1,631 



1. Excludes fee-for-service payments made to physicians for providing services to non-enrolled patients and for providing non-listed services to enrolled patients 

2. Excludes traditional fee-for-service payments before the 10% top-up 



an alternate funding arrangement receives in total, 
because the alternate funding arrangements allow 
some or all payments to be made directly to the 
physician group for distribution at its discretion. 

Ministry staff informed us that the Family 
Health Group (FHG) arrangement was initially 
designed to pay more than the traditional fee-for- 
service amount in order to encourage physicians 
to join an alternate funding arrangement. It pays 
physicians 110% of the traditional fee-for-service 
amount. For other alternate funding arrangements, 
such as Family Health Organizations (FHOs) and 
Family Health Networks (FHNs), base capitation 
payments, as well as the continued fee-for-service 
payments for non-listed services and for services 
provided to non-enrolled patients, were designed 
to be expenditure-neutral — that is, costing neither 
more or less than before. The Ministry indicated 
that additional payments, such as bonuses and 
incentives, are included in all alternate funding 
arrangements to compensate and reward physicians 
for providing high-quality comprehensive care. This 
in turn makes these arrangements more lucrative 
for physicians than being paid through the trad- 
itional fee-for-service model. 

Payments to family physicians under the alter- 
nate funding arrangements are complicated. For 
example, in the 2009/10 fiscal year, there were 
up to 42 types of payments made to physicians 
working in FHGs, and 61 types of payments made 
to physicians working in FHOs. (Selected payment 
types are shown in Figure 1.) Most of these pay- 
ments are incentives in the form of premiums and 



bonuses. For example, under both FHG and FHO 
arrangements, there are eight types of incentives 
offered to physicians for enrolling patients, and 
12 types of incentives to physicians for providing 
preventive-care activities, such as vaccinations and 
cancer-screening tests. 

Further, the basis for determining the amount 
of payment differs among the various payment 
types. For example, some payments (such as 
capitation) are based on the number (and certain 
other characteristics) of enrolled patients. Other 
payments (such as shadow billing and after-hours 
incentives) are based on the number of actual 
services provided to enrolled patients. Still other 
payments (such as administration fees, continuing 
medical education fees, and fees for being on call 
for the Telephone Health Advisory Service) are on 
a per-physician basis. Physicians can also receive 
bonuses for services their patients receive from 
other health-care professionals (for example, flu 
vaccines, mammograms, and colorectal screening) 
as long as a certain percentage of the physicians' 
patients receive the services. 

The many different payment types make alter- 
nate funding arrangements more difficult for the 
Ministry to administer and monitor. We reviewed 
the capitation and access bonus payments, which 
together constitute more than 50% of payments 
made through the alternate funding arrangements. 
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Capitation Payments 

Physicians participating in FHNs and FHOs gener- 
ally receive two types of capitation payments (base 
capitation and comprehensive care capitation) for 
every enrolled patient, whether or not the patient 
visits them in a given year. As FHNs and FHOs 
receive both of these payments for all enrolled 
patients, we questioned whether the capitation 
rates could be combined. 

We noted that the capitation rates in Ontario, 
similar to those in other Canadian provinces, are 
based only on the patient's age and sex, and do not 
consider the patient's health condition and health- 
care needs. In comparison, England's capitation 
payments to physicians consider, in addition to the 
patient's age and sex, the patient's health condi- 
tion (morbidity), and New Zealand's consider how 
frequently the patient uses health-care services. 
Linking a patient's health condition and need for 
health-care services to the capitation funding for 
that patient could encourage physicians to treat 
hard-to-care-for patients and possibly eliminate the 
need to offer physicians additional premiums for 
seeing such patients. The Physician Services Agree- 
ment between the Ministry and the Ontario Medical 
Association indicated that a working group would 
be established to report to the Ministry by December 
2011 on updating the capitation methodology. 

The base capitation rate covers patient servi- 
ces listed in the alternate funding arrangement 
contracts. The capitation payment for each type of 
alternate funding arrangement includes a differ- 
ent list of patient services. For example, the FHO 
arrangement covers almost 120 services and has a 
capitation rate that can be up to 40% higher than 
the FHN arrangement, which covers only half that 
many services. The FHO arrangement includes 
services such as house-call assessments, palliative 
care, and single-injection chemotherapy, which are 
not covered by the FHN contract. Physicians under 
both arrangements still bill the Ministry on a fee- 
for-service basis for "non-listed" services provided 
to their enrolled patients. Our analysis indicated 



that, despite FHOs having twice as many services 
listed in their arrangement as FHNs, in the 2009/10 
fiscal year, 27% of physician services provided to 
patients enrolled with an FHO, as compared to 
32% of services provided to patients enrolled with 
an FHN, were non-listed and billed to the Ministry 
as fee-for-service claims. The Ministry paid FHO 
and FHN physicians an additional $72 million and 
$13 million, respectively, for the non-listed services. 

We further noted that almost 30% of the fee-for- 
service claims for non-listed services were for flu 
shots and Pap-smear technical services (that is, the 
cost of supplies used to perform the test). However, 
the Ministry had not analyzed which services would 
be most cost-effective to list in the alternate funding 
arrangements, or whether it would be beneficial to 
have capitation payments include most if not all of 
the more routine or most common medical services. 

Access Bonus 

Family physicians participating in an FHO or 
FHN whose enrolled patients seek primary-care 
services elsewhere lose a portion of their practice's 
total base capitation, known as the access bonus. 
This amount is calculated monthly. The loss is 
equal to the fee-for-service payments made by the 
Ministry to the physician who treated the patient, 
to a maximum penalty of 18.59% of the practice's 
total base capitation for FHOs and 20.65% for FHNs. 

In the 2009/10 fiscal year, about 1.3 million 
patients who were enrolled with an FHO or FHN 
made 3.7 million visits to family physicians outside 
their practice. (This represented almost 30% of 
patients enrolled under these arrangements, and 
almost 20% of the times patients sought care from a 
family physician.) As a result, capitation payments 
for FHO and FHN physicians were reduced by 
$54 million and $4 million, respectively. Ministry 
data for the 2010 calendar year indicated that 140 
FHOs and six FHNs were penalized the maximum 
amount in at least one month, with 25 FHOs penal- 
ized the maximum every month. Because these 
practices had reached the maximum percentage 
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penalty, the Ministry could not recover the addi- 
tional $11 million that it paid for their enrolled 
patients who sought care elsewhere. Therefore, the 
Ministry paid twice for these services — once through 
capitation payments to FHOs and FHNs and again 
through fee-for-service payments to other physicians. 

We also noted that there is no deduction from 
the access bonus if an enrolled patient visits an 
emergency department for non-emergency care. 
Based on our analysis of claim submissions for 
the 2009/10 fiscal year, we found that although 
non-enrolled patients visited emergency depart- 
ments about 10% of the time for their medical care, 
enrolled patients visited them slightly less often 
(about 7% of the time), with more than 40% of 
enrolled patients' visits being for non-urgent care. 



RECOMMENDATION 4 



To facilitate the administration of the current 
complex alternate funding arrangements for 
family physicians, the Ministry of Health and 
Long-Term Care (Ministry) should consider 
reducing the number of arrangements and sim- 
plifying the types of payments. Further, to better 
ensure that the alternate funding arrangements 
are cost-effective, the Ministry should: 

• review the fee-for-service payments to phys- 
icians for services not covered by the annual 
capitation payment, and determine whether 
significant savings may be possible by having 
them covered by the capitation payment; and 

• consider negotiating a reduction in capita- 
tion payments for patients who never or 
seldom see the physician they are enrolled 
with, as well as a further reduction in capita- 
tion payments to better reflect the cost of 
non-emergency services that patients obtain 
from physicians who are not part of the prac- 
tice they are enrolled with. 

| MINISTRY RESPONSE 

The Ministry agrees that the current array of 
alternate funding arrangements and the com- 



plexity of the payment elements under each of 
these arrangements are complex. The Ministry 
supports the concept of simplified agreements 
and the reduction of the number of alternate 
funding arrangements. 

However, the Ministry recognizes the 
advantage of using specific bonus and premium 
payments to encourage the delivery of specific 
services and/or activities that are a priority for 
the Ministry. The continuation of bonuses and 
premiums allows funding to be targeted toward 
specific programs of interest. 

The Ministry will undertake a review of the 
payment elements under the existing alternate 
funding arrangements to identify opportunities 
for simplification. 

The Ministry supports a review of existing 
included and excluded services under the 
Family Health Network and Family Health 
Organization alternate funding arrangements, 
to determine whether changes are required to 
better reflect the full range of comprehensive 
primary-care services. 

The Ministry is currently in discussion with 
the Ontario Medical Association (OMA) to 
consider changes to the capitation payment to 
reflect patient acuity/complexity. The Ministry 
will approach the OMA to express its interest in 
having utilization-based modifiers considered in 
addition to modifiers based on patient acuity/ 
complexity. 

The Ministry and the OMA, through the 
Primary Health Care Committee, are currently 
conducting a policy review of the access bonus 
payment. This policy review will consider cur- 
rent services contributing to the access bonus, 
contract provisions related to the negative 
access bonus, and provisions to ensure that 
there are not unintended incentives to encour- 
age the use of emergency departments outside 
available office hours. 
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MONITORING 

Most provinces with alternate funding arrange- 
ments for family physicians, including Ontario, 
request physicians to shadow bill for services 
included in their contracts in order to obtain infor- 
mation on the frequency and nature of physician 
services being provided. However, the Ministry 
informed us that it has not analyzed shadow billing 
claims to determine the number of patients seen 
or the clinical services provided by physicians paid 
through alternate funding arrangements. The Min- 
istry informed us that although there is no reason 
to believe that physicians would forgo income avail- 
able through shadow billing, it has no assurance 
that physicians actually do shadow bill for all patient 
services rendered. Therefore, the shadow billing 
data may not reflect all patient services provided. 

As well, although the Ministry has some cost 
information, it has not tracked the full cost of each 
funding arrangement since the 2007/08 fiscal 
year. Therefore, the Ministry cannot compare the 
amounts paid for the services delivered by family 
physicians among the different alternate funding 
arrangements or compare the payments made to 
the medical services being delivered. It also cannot 
compare amounts paid under alternate funding 
arrangements to the amounts paid to family phys- 
icians who are remunerated solely through fee-for- 
service billings. 

Further, some aspects of the alternate funding 
arrangement contracts are difficult to monitor, such 
as whether physicians invite all of their patients to 
enrol with the group or whether physicians refuse 
to enrol certain patients when they switch to an 
alternate funding arrangement. This information 
would assist the Ministry in determining whether 
the more costly alternate funding arrangements are 
meeting its goals. 

In addition, without good information on the 
relative costs and service levels being provided, it is 
more difficult for the Ministry to effectively negoti- 
ate with the Ontario Medical Association to make 
changes to alternate funding arrangements that 



would better enable the Ministry to meet its goals, 
or to consolidate the current funding arrangements. 
The Ministry informed us that its initial focus was 
on encouraging as many family physicians as pos- 
sible to join an alternate funding arrangement, in 
order to improve patient access, provide income 
predictability for physicians, and better predict 
Ministry expenditures. The Ministry indicated that 
it plans to increase its monitoring of these arrange- 
ments now that more than 60% of family physicians 
are participating. 



RECOMMENDATION 5 



To provide the Ministry of Health and Long- 
Term Care (Ministry) with information that 
would facilitate better monitoring of the bene- 
fits and costs of each alternate funding arrange- 
ment for family physicians, the Ministry should: 

• periodically review shadow billing data 
to determine the frequency and nature of 
services provided by physicians in each 
arrangement; 

• track the total amount paid to physicians 
participating in each arrangement; and 

• track the average amounts paid to each 
physician both for reasonableness and for 
the purposes of comparing them to physician 
compensation under the traditional fee-for- 
service funding model. 



The Ministry supports the recommendations 
for improved monitoring of alternate funding 
arrangements. The Ministry recognizes that 
increased monitoring of the contracts will better 
ensure the achievement of the Ministry's goals 
and objectives in relation to the alternate fund- 
ing arrangements, and the Ministry will imple- 
ment the monitoring activities identified in the 
recommendations, as follows: 
• introduce a process whereby shadow billed 
services by physicians in the Family Health 
Network and Family Health Organization 
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alternate funding arrangements will be 
reviewed to ensure that the volume and 
nature of the services are consistent with 
expected service levels and the services 
included in the contract; 
• track total payments by model annually; and 
o establish payment tracking at a physician 
level to compare base rate payments in the 
Family Health Network and Family Health 
Organization alternate funding arrange- 
ments with the fee-for-service equivalent, by 
using shadow billing claims data. 



Chapter 3 



Ministry of Health and Long-Term Care 

Section 

3.07| Funding Alternatives for 
Specialist Physicians 




1 




Physicians may provide specialized services in 
over 60 areas, including cardiology, gynecology, 
orthopaedics, pediatrics, and emergency services. 
These specialists work in various settings, including 
hospitals and their own offices. 

In the 1990s, the Ministry of Health and Long- 
Term Care (Ministry) introduced funding alterna- 
tives (known as alternate funding arrangements) 
for some specialist physicians to encourage them to 
provide certain services, such as academic services 
(including training new physicians and conducting 
research) and working in remote areas of the 
province. Before this, the Ministry paid specialist 
physicians on a fee-for-service basis for the differ- 
ent clinical services involved in diagnosing and 
treating patients, but did not compensate specialists 
for these other services. In 1999, the Ministry also 
introduced specialist alternate funding arrange- 
ments for physicians, generally family physicians, 
for providing emergency services in hospitals. Most 
of the specialists paid through alternate funding 
arrangements may also bill the Ministry on a fee- 
for-service basis for patient care provided outside 
the arrangement. 

Alternate funding arrangements are contractual 
agreements between the Ministry, a group of phys- 



icians, and in most cases the Ontario Medical Asso- 
ciation (the organization that bargains on behalf 
of physicians in Ontario) and may include other 
organizations such as hospitals and universities. 
Alternate funding arrangements for specialists are 
also subject to provisions in the physician services 
agreements between the Ministry and the Ontario 
Medical Association, which have been negotiated 
every four years since 2000. 

In the 2009/10 fiscal year, the Ministry paid 
almost $1.1 billion, as shown in Figure I, under 
specialist alternate funding arrangements to more 
than 9,000 physicians. This represents about 17% 
of the $6.3 billion the Ministry paid to all specialists 
that year. As of March 31, 2010, 50% of the almost 
13,000 specialists in the province and more than 
90% of the 2,700 emergency department phys- 
icians were paid, at least in part, through a special- 
ist alternate funding arrangement. 



Audit Objectives and Scope 



This year, our Office performed two audits on 
funding alternatives (known as alternate funding 
arrangements) for physicians. The audit discussed 
in this section focused on the arrangements for 
specialist physicians, and the audit in Section 3.06 
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Figure 1: Number of Physicians Participating in 
Specialist Alternate Funding Arrangements and 
Associated Payments, by Agreement Type 

Source of data: Ministry of Health and Long-Term Care 





#of 
Physicians 
as of 


Payments 
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Fiscal Year 


Agreement Type 


March 31, 2010 


($ million) 


academic 


1,234 


268 


comprehensive 1 


Academic Health 


3,692 


242 


Science Centres 2,3 


emergency 
departments 


2,653 


315 


northern specialists 3 


280 


39 


other 


1,181 


208 


Total 


9,040 


1,072 



1. Unique alternate funding arrangements for academic services, including 
training new physicians and conducting research. 

2. Standard alternate funding arrangement for academic services, including 
training new physicians and conducting research. 

3. Excludes fee-for-service payments to participating physicians for clinical 
services. 



focused on those for family physicians. Our audit 
objective was to assess whether the Ministry of 
Health and Long-Term Care (Ministry) has imple- 
mented systems and processes to monitor and 
assess whether alternate funding arrangements 
provide Ontarians with timely access to specialist 
physicians in a cost-effective manner. Ministry 
senior management reviewed and agreed to our 
objectives and associated audit criteria. 

Given the number of different alternate funding 
arrangements for specialists, our audit focused pri- 
marily on arrangements with academic physicians 
(whose responsibilities generally include training 
new physicians and conducting research) and 
emergency department physicians, and to a lesser 
extent on payments to specialists working in North- 
ern Ontario. Contracts with these groups currently 
encompass over 85% of physicians who participate 
in a specialist alternate funding arrangement. 

Our audit work was conducted primarily at the 
Ministry's Negotiations Branch in Toronto, which 
is responsible for managing the specialist physician 



contracts, as well as at other ministry branches in 
Toronto. In conducting our audit, we reviewed rel- 
evant files, systems, and administrative policies and 
procedures; interviewed appropriate ministry staff; 
and reviewed relevant research from Ontario and 
other jurisdictions. We also reviewed data received 
from the Ministry's Ontario Health Insurance Plan 
database. We did not rely on the Ministry's internal 
audit service team to reduce the extent of our audit 
work, because it had not recently conducted any 
audit work on alternate payment arrangements for 
specialists or emergency department physicians. 



Summa 



- 



Payments made under alternate funding arrange- 
ments for specialists and emergency department 
physicians increased by more than 30% from the 
2006/07 fiscal year to almost $1.1 billion in the 
2009/10 fiscal year, or more than 10% per year, 
similar to the increase in payments to all special- 
ists during this time. By 2009/10, payments made 
under alternate funding arrangements accounted 
for about 17% of all payments to specialists and 
emergency department physicians. However, the 
Ministry has conducted little formal analysis of 
whether the expected benefits of these alternate 
funding arrangements, such as improving patient 
access, have materialized or have been cost- 
effective. For instance, payments to emergency 
department physicians increased by almost 40% 
between the 2006/07 and 2009/10 fiscal years, 
while the number of physicians working in emer- 
gency departments increased by only 10% and the 
number of patient visits increased by only 7%. 

We also noted that although the Ministry 
indicated that it performed a cost/benefit analy- 
sis before it entered into any alternate funding 
arrangements, it was unable to provide us with 
any such analysis relating to the arrangements 
that most of the physicians participated in. Addi- 
tionally, the relative complexity of the different 
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arrangements and the relative scarcity of perform- 
ance measures in the contracts have made it diffi- 
cult for the Ministry to effectively monitor both the 
accuracy of payments being made and the extent to 
which physicians have actually provided the servi- 
ces expected in their contracts. 

Some of our more significant observations are as 
follows: 

• The Ministry has made progress in imple- 
menting standard contracts for most special- 
ists, and these contracts are now in place for 
more than 70% of physicians participating in 
specialist alternate funding arrangements. 

• The Ministry does not track the total amounts 
paid to physicians participating in Academic 
Health Science Centre (AHSC) and northern 
specialist alternate funding arrangements, 
and therefore cannot readily perform any sub- 
sequent assessment of the cost-effectiveness 
of the alternate funding approach and also 
cannot compare the income of physicians paid 
through these arrangements to the income 

of physicians performing similar work but 
paid under the traditional fee-for-service 
arrangement. 

• The alternate funding arrangement contracts 
generally do not contain measures by which 
the Ministry can assess the extent to which the 
objectives necessitating the alternate fund- 
ing arrangement, such as improving patient 
access and advancing innovation in medicine, 
have been achieved. 

• There are numerous types of payments and 
various premiums that specialists can earn, 
making contract- and payment-monitoring 
difficult for the Ministry. For example, for aca- 
demic services (including training new phys- 
icians and conducting research), there were 
up to nine different categories of payments 
under AHSC contracts and up to 14 categories 
under academic comprehensive contracts. 

• Ten AHSCs received "specialty review fund- 
ing" totalling $19.7 million in the 2009/10 



fiscal year as an interim measure to alleviate 
immediate human resource challenges in five 
specialty areas. However, similar temporary or 
interim funding has been given annually since 
2002. 

• In May 2007, the Ministry obtained permis- 
sion from 234 northern specialists to collect 
information on each physician's income from 
provincial government-funded sources. The 
Ministry paid these physicians $15,000 each. 

• As a means to monitor whether specialists 
funded under academic contracts have met 
their contract obligations, the Ministry pro- 
vided them with a checklist to self-evaluate 
their performance in this regard. However, 
the Ministry does not request the results of 
this self-evaluation, and it does only minimal 
other monitoring of these specialists to ensure 
that they are providing the level of service 
outlined in their contracts. 

We also noted instances where the Ministry 
chose not to recover its overpayments to physicians. 
Our observations in this regard included the 
following: 

• The Ministry has a good process in place to 
identify overpayments to emergency depart- 
ment physicians and found $3.9 million in 
overpayments from the 2005/06 fiscal year to 
the 2009/10 fiscal year. However, even though 
the physicians at these emergency depart- 
ments worked fewer hours than they were paid 
for, the Ministry did not attempt to recover 
any of the overpaid funds because it was 
concerned this would negatively affect patient 
wait times at these emergency departments. 

• In April 2008, the Ministry paid over $15 mil- 
lion to 292 physicians who signed a document 
indicating their intent to join a northern 
specialist alternate funding arrangement. 
However, 11 of the physicians, who were paid 
a total of $617,000, did not subsequently join 
an alternate funding arrangement yet were 
allowed by the Ministry to keep the funding. 
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• The Ministry's review of service levels pro- 
vided by AHSCs during the 2007/08 and 
2008/09 fiscal years indicated that 40% 
generally had at least one specialty area that 
did not meet the contracted service-level 
requirements. However, no attempt was made 
to recover these overpayments nor was any 
adjustment to future funding levels made. 

OVERALL MINISTRY RESPONSE 

The Ministry welcomes the report from the 
Office of the Auditor General regarding 
alternate funding arrangements for specialist 
physicians. These arrangements were founded 
to address specific concerns, including sus- 
taining or improving access to health-care 
services for all Ontarians regardless of income, 
geography, or other barriers to access. In this 
regard, the arrangements were often aimed at 
communities, services, and programs where the 
volume-driven fee-for-service model did not fit. 
To this end, the Ministry funds the majority of 
emergency departments, hospital-based north- 
ern specialists, and medical training, research, 
and innovation activities through alternate 
funding arrangements. The anticipated benefits 
of the arrangements are timely patient access to 
health services and reduced wait times, a reduc- 
tion in travel costs, decreased morbidity and 
mortality, a reduction in hospitalizations and 
hospital-related costs, and a new generation of 
well-trained specialist physicians. 

The Ministry appreciates the comments from 
the Auditor General about ongoing cost/benefit 
analyses of the alternate funding arrangements. 
Although the cost of these arrangements is off- 
set in part by a reduction in fee-for-service pay- 
ments, the measure of cost-effectiveness is not 
only as compared to fee-for-service, but must 
also take into account benefits associated with 
a range of health determinants over the long 
term, including access to care. The Ministry sup- 
ports the need for further research in this area. 



The Ministry also supports the need for 
clearly defined reporting expectations and 
meaningful performance measures and targets. 
As the Auditor General has noted, the Ministry 
has made progress on implementing standard 
contracts to reduce the complexity among 
agreement types. Furthermore, the Ministry 
is engaged in continuing this process through 
continual review and modernization of existing 
agreements to ensure that existing agreements: 

• continue to address the Ministry's objectives; 

• are in compliance with established protocols 
and processes; 

• include appropriate performance monitoring 
and reporting provisions; and 

• include appropriate, timely, and docu- 
mented corrective actions. 



servations 



OVERVIEW 



Like many other Canadian jurisdictions, Ontario has 
alternate funding arrangements for specialists. The 
Ministry's goals for these arrangements include: 

• maintaining and enhancing the academic 
activities of physicians (for example, training 
medical students and conducting research) ; 

• enhancing income predictability and stability 
for physicians; and 

• increasing the recruitment and retention of 
physicians in underserviced areas. 

At the time of our audit, there were 10 types 
of specialist alternate funding arrangements, 
including arrangements for academic specialists; 
emergency department physicians; and special- 
ists working in Northern Ontario. A specialist 
arrangement may fund an individual department 
in one hospital, or it may cover a range of services 
provided by all the physicians at a hospital. Prior to 
2004, groups of physicians contacted the Ministry 
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to establish and participate in alternate funding 
arrangements. These arrangements represent 
over 80% of participating physicians. Since 2004, 
groups of physicians may initially contact either 
the Ministry or the Ontario Medical Association 
to propose new arrangements, or arrangements 
may be proposed directly by the Ontario Medical 
Association to the Ministry. Arrangements are then 
negotiated between the Ministry and the Ontario 
Medical Association. Physician groups generally 
have a governing organization (sometimes called a 
governance group), whose responsibilities include 
deciding how payments to the group will be allo- 
cated among participating physicians. 

The specialist alternate funding arrangements 
are primarily managed by the Specialist Physician 
Contracts Unit in the Ministry's Negotiations 
Branch. Other branches within the Ministry are also 
involved in helping administer the contracts. These 
include the Financial Management Branch, which 
is responsible for processing physician payments 
and conducting financial forecasting and reporting; 
the Health Data Branch, which is responsible for 
collecting statistics relating to physician counts, 
conducting trend analyses, and calculating certain 
payments; the Registration and Claims Branch, 
which is responsible for processing physician regis- 
trations; and the Health Solutions Delivery Branch, 
which is responsible for developing information 
systems to support new types of payments or chan- 
ges in payment rates. 

CONTRACTING WITH SPECIALISTS 

For most of the arrangements, either the Ontario 
Medical Association or a specialist group that 
was interested in receiving compensation for 
services not funded through fee-for-service pay- 
ments approached the Ministry requesting that 
an alternate funding arrangement be established, 
such as for training and research. They may also 
have requested funding for other reasons, such as 
increasing physician income when patient volume 
in a region is too low to provide a full-time specialist 



with a fee-for-service income level similar to what 
he or she would earn in other parts of the province. 
The Ministry generally is not approached about 
establishing an alternate funding arrangement for 
specialist groups that do not have concerns about 
the equity of their compensation levels, such as 
ophthalmologists, cardiologists, and radiologists. 

The Ministry indicated that it reviews submitted 
proposals outlining why a physician group should 
receive alternate funding; compares the costs of 
the proposed alternate funding arrangement with 
the historical fee-for-service costs; and assesses the 
proposed benefits, such as improved patient access 
to care. We requested the Ministry's analyses for 
various specialist alternate funding arrangements, 
including arrangements for emergency depart- 
ments, Academic Health Science Centres (AHSCs), 
and northern specialists, but it was unable to locate 
its analyses for these funding arrangements. How- 
ever, the Ministry was able to locate a cost estimate 
prepared by the Ontario Medical Association for 
AHSCs. Based on this estimate, payments were 
expected to increase by 33%. As well, the Ministry 
was able to locate its cost estimates for two recent 
emergency department contracts. These estimates 
indicated that payments for physician services were 
expected to increase by 32% and 60%, respectively. 
The Ministry indicated that the benefits of these 
arrangements were expected to include improved 
patient access to care. 

If the Ministry decides to pursue the alternate 
funding arrangement, it begins negotiations, 
which are generally with the specialist group of 
physicians, the Ontario Medical Association, and 
often the hospital at which the specialists provide 
services. In the case of specialists who train med- 
ical students, a university may also be part of the 
negotiations. As a result of these negotiations, the 
Ministry has developed standard contracts for most 
of the alternate funding arrangements, including 
those involving emergency departments, AHSCs, 
and northern specialists. For the few non-standard 
funding arrangements (for example, academic 
comprehensive agreements, which were developed 
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prior to the standardized AHSC contracts), each 
specialist group receiving funding under the same 
type of plan negotiates a unique contract with the 
Ministry. As of March 31, 2011, the Ministry had 
almost 250 agreements with specialist groups, as 
shown in Figure 2. The Ministry informed us that it 
intends to develop standard contracts for all plans 
in the future. 

The contracts generally stipulate the amount of 
funding the specialists will receive, the service levels 
that the specialists must provide, recruitment and 
retention mechanisms for new specialists, and infor- 
mation that specialists must report to the Ministry. 
As well, the contracts usually include objectives such 
as improving patient access; supporting the clinical 
training needs of medical students, physicians, and 
other health-care providers; and advancing innova- 
tion in medicine. However, while the AHSC arrange- 
ment has more than 20 performance measures, the 
other arrangements generally do not have any. The 
Ministry had not used the measures in the AHSC 
contract to determine to what extent the objectives 
necessitating the alternate funding arrangement 
had been achieved. Further, the measures in the 
AHSC arrangement did not include the number of 
patients seen or wait times to access care. These 
measures would assist the Ministry in assessing 
whether the service levels and overall intent of the 
arrangements were being met. 

Most specialist physicians who participate in 
an alternate funding arrangement are required to 
sign a form to indicate their acceptance of the con- 



Figure 2: Number of Contracts by Agreement Type, 
as of March 31, 2011 

Source of data: Ministry of Health and Long-Term Care 
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tract's terms. By signing such a form, a physician is 
agreeing to, among other things, provide services 
in accordance with the contract and not bill the 
Ontario Health Insurance Plan (OHIP) for these 
services except as provided for under the contract. 
Some contracts require participating physicians to 
sign the form before they begin to provide services; 
other contracts state that they must sign the form 
within 30 days of beginning to provide services; 
and still other contracts are silent regarding when 
the forms must be signed. For contracts tested 
where we would expect to have seen physician- 
signed forms, we found that only 30% of physicians 
signed consent forms before they began providing 
services. An additional 42% signed consent forms 
after they began providing services, and the Min- 
istry did not have consent forms for the remaining 
28%. Without a signed consent form, there is a 
risk that physicians may not fully understand their 
obligations and, for example, not provide the level 
of patient services required under the contract. 



RECOMMENDATION 1 



To help ensure that compensation arrangements 
for specialists meet the Ministry of Health and 
Long-Term Care's goals and objectives in a finan- 
cially prudent manner, the Ministry should: 

• assess and document the anticipated costs 
and benefits of each alternate funding 
arrangement, compared to the standard 
fee-for-service compensation method, before 
entering into a formal agreement; 

• incorporate specific performance measures 
into the contracts, such as the number of 
patients to be seen or the wait times to 
access care, to enable the Ministry to peri- 
odically assess what benefits are received for 
the additional cost of the arrangement; and 

• require physicians to sign that they agree to 
the terms of the contract before commen- 
cing participation in an alternate funding 
arrangement. 
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The Ministry supports this recommenda- 
tion, and as it moves forward to negotiate or 
renegotiate alternate funding arrangements 
it will work toward full compliance with this 
recommendation. 

In recent years, alternate funding arrange- 
ments have been negotiated as part of the 
overall Physician Services Agreement discus- 
sions with the Ontario Medical Association. 
The alternate funding arrangements negotiated 
and implemented as part of this process are 
developed to ensure that the goals and stra- 
tegic priorities of the Ministry and the Ontario 
government are met. These goals and priorities 
include ensuring access to high-quality health 
care for all Ontarians and providing specialist 
services in underserviced communities. The 
Ministry will continue to compare the initial 
cost of each alternate funding arrangement to 
the fee-for-service compensation method before 
entering into a formal agreement. 

The Ministry supports the principle of incor- 
porating specific performance measures into the 
arrangements and is committed to improving 
how it demonstrates measurable results as it 
meets its goals and priorities in a cost-effective 
manner. All agreements negotiated or renegoti- 
ated with specialists will have the roles, 
responsibilities, accountability relationships, 
and obligations of all parties clearly defined and 
documented. In addition, the Ministry will work 
toward implementing reporting expectations 
with meaningful performance measures and 
targets. 

The Ministry requires all participating phys- 
icians to sign an agreement before commencing 
participation in an alternate funding arrange- 
ment and will ensure full compliance with this 
obligation. 



PAYING SPECIALISTS 

Total ministry payments under both fee-for-service 
and alternate funding arrangements to all spe- 
cialists and emergency department physicians 
increased by over 25% from $5 billion in the 
2006/07 fiscal year to over $6.3 billion in the 
2009/10 fiscal year, the most recently available 
data for total payments to these groups. Some- 
what similarly, payments made under alternate 
funding arrangement contracts for specialists and 
emergency department physicians increased by 
over 30% during the same period, from more than 
$800 million in the 2006/07 fiscal year to almost 
$1.1 billion in the 2009/10 fiscal year. 

Payments to specialists under the alternate fund- 
ing arrangements are complicated, because there 
are numerous types of payments and various pre- 
miums that specialists can earn. Figure 3 outlines 
selected types of payments. 

Figure 4 provides further information about 
how physician compensation is determined under 
selected specialist alternate funding arrangements. 

Academic Physicians 

Academic specialists represent more than half of 
the specialists participating in alternate funding 
arrangements. There are two main arrangements 
for academic specialists: 

• Academic Health Science Centres (AHSCs) — 
a standardized arrangement introduced in 
2003 to support academic physicians work- 
ing at AHSCs, which are formed through 

an agreement between a university with a 
medical school, a hospital where medical 
students are trained, and physicians that work 
at both. In the 2009/10 fiscal year, 3,700 
physicians received $242 million under this 
arrangement. 

• Academic comprehensive — unique agree- 
ments established prior to the introduction 
of the AHSC arrangements involving three 
hospitals and the associated universities, 
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Figure 3: Selected Types of Payments under Specialist Alternate Funding Arrangements 

Prepared by the Office of the Auditor General of Ontario 


1 Type of Payment 


Description 


base funding 


A lump sum paid to specialist groups for providing a collection of services 


fee-for-service 


Physicians bill OHIP and are paid an established fee for each service provided to a patient 


shadow billing 


Physicians who receive base funding can bill OHIP and be paid a percentage of the established fee for 
each service provided to a patient 


premiums 


Additional payments to physicians to provide specific services, such as patient care on weekends 


administration 


Amounts paid to specialist groups for administering alternate funding arrangements 



to support academic physicians. In the 
2009/10 fiscal year, 1,200 physicians received 
$268 million under this arrangement. 
Figure 4 highlights some of the significant differ- 
ences between these two payment arrangements. 

Up to nine types of payments were made under 
each AHSC contract, and up to 14 types of pay- 
ments were made under each academic compre- 
hensive contract, including payments for the items 
shown in Figure 3. Based on our testing of these 
payments primarily in the 2009/10 fiscal year, we 
noted that: 

• The Ministry did not have documentation to 
support whether the base funding amount 
paid under the three academic comprehensive 
contracts in the 2009/10 fiscal year was accur- 
ate. A significant portion of the base funding 
amount is based on physicians' highest 12 
consecutive months of OHIP billings before 
joining the alternate funding arrangement. 
We noted that for 2009/10, base payments to 
one hospital exceeded the contract amount by 
$2.1 million. Ministry staff informed us that 
the majority of the difference was likely due to 
physicians entering and leaving the academic 
group. However, the Ministry had no informa- 
tion on who had joined or left the groups, 
which would be needed to substantiate the 
amount that was paid. 

• Funding to recruit recently graduated phys- 
icians or physicians new to the province began 
under the academic comprehensive contracts 
in the 2008/09 fiscal year. However, one 



academic physician group was already receiv- 
ing $575,000 annually for the recruitment of 
physicians, having negotiated that payment as 
part of its base funding. This physician group 
received additional funding for recruitment 
activities after the recruitment funding was 
introduced in 2008/09, including an addi- 
tional $495,000 in the 2009/10 fiscal year, 
as the funding was available to all groups 
including the group that was already receiv- 
ing recruitment funding. 

• In the 2007/08 fiscal year, $8.5 million in 
recruitment funding for AHSC physicians 
was allocated to their governance groups to 
distribute as they saw fit. Based on reports 
received by the Ministry, $3.2 million of this 
funding was spent on recruiting physicians. 
The Ministry had no information on how the 
remaining $5.3 million was spent. Similar 
issues were not noted in subsequent years. 

• Ten hospitals received "specialty review fund- 
ing" totalling $19.7 million in the 2009/10 
fiscal year to, according to the Ministry, "serve 
as an interim measure to alleviate immediate 
human resource challenges" in five specialty 
areas. Although it was indicated that it was an 
"interim" measure, similar temporary fund- 
ing actually had been in place annually since 
2002. The Ministry informed us that a formal 
review was done in 2002 that determined 
that there was a funding shortfall in these five 
specialty areas, but the Ministry was unable 
to provide any documentation relating to this 
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review. The Ministry also indicated that the 
funding has been periodically reviewed. The 
most recent review was in 2009 and involved 
a two-day consultation with funding recipi- 
ents. The Ministry concluded on the basis of 
discussions with the recipients that the fund- 
ing was having a positive impact and would 
continue unchanged until further review. 
• Clinical repair funding provides academic 
physicians with additional income to make 
their income levels comparable to those of 
non-academic physicians, who generally have 
time to see and bill for more patients. Since 
its introduction in the 2007/08 fiscal year, 
clinical repair funding has been calculated 
annually based on what similar non-academic 
specialists billed OHIP in the 2006/07 fis- 
cal year. However, the Ministry had no 
documented analysis of whether the clinical 
repair funding amount in the 2010/11 fiscal 
year made the income of academic special- 
ists reasonably comparable to non-academic 
specialist incomes. The Ministry indicated 
that it commenced a review of AHSC funding 
in 2010, which includes a review of clinical 
repair funding. The Ministry expects to com- 
plete this review by December 2011. 

Emergency Department Physicians 

Funding to the province's more than 145 emergency 
departments is intended to provide for around-the- 
clock emergency services. Between the 2003/04 
and 2006/07 fiscal years, total ministry funding 
for emergency department (ED) physician services 
increased by almost 35%, as shown in Figure 5. Sim- 
ilarly, from the 2006/07 fiscal year to the 2009/10 
fiscal year, payments for ED physician services also 
increased by almost 40% in total, although the num- 
ber of physicians who worked in emergency depart- 
ments increased by only 10%, and the number of 
patient visits to emergency departments increased 
by only 7% during the same period. The Ministry 
stated that 10 additional emergency departments 



joined an alternate funding arrangement during this 
time and that alternate funding arrangements usu- 
ally result in increased payments to physicians. The 
Ministry also indicated that the primary goals of the 
ED alternate funding arrangement were to ensure 
that emergency departments remain open 24 hours 
a day, seven days a week, and maintain a stable 
workforce of physicians. 

For the 2009/10 fiscal year, payments for the 
services of ED physicians participating in an alter- 
nate funding arrangement consisted primarily of 
$268 million in base funding with shadow billing 
(that is, physicians are paid 25% of the established 
fee-for-service amount for submitting data to 
OHIP on patient services provided) as well as 
additional premiums and other payments totalling 
$47 million. 

Funding for ED physicians participating in the 
workload model, under which more than one phys- 
ician may be working in the emergency department 
at the same time, is based on patient acuity (that 
is, the urgency of care required by the patient) as 
well as on patient volume in the previous year. The 
Ministry inputs this information into a formula 

Figure 5: Ministry Funding for All Emergency 
Department Physician Services, 2003/04-2009/10 
($ million) 

Source of data: Ministry of Health and Long-Term Care 
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to determine the number of hours ED physicians 
are required to work to meet patient needs. The 
formula was developed based on research com- 
missioned by the Ministry. The Ministry uses 
these hours and an hourly rate to determine the 
funding to be provided annually to each group 
of ED physicians. The Ministry indicated that the 
established hourly rate was initially developed 
in the 1999/2000 fiscal year by a working group 
consisting of representatives from the Ministry 
the Ontario Medical Association, and the Ontario 
Hospital Association. 

The Ministry indicated that the current fund- 
ing levels for ED groups under the 24-hour model 
(where only one ED physician is on duty at a time) 
were set in the 2006/07 fiscal year, based on nego- 
tiations between the Ministry, the Ontario Medical 
Association, and the Ontario Hospital Associa- 
tion. Since then, funding for ED groups under the 
24-hour model has generally been based on patient 
volumes for the previous calendar year. 

We noted that overpayments to ED physician 
groups were not being recovered. For example: 
• Under the Ministry's contract with physicians 
participating in the workload model, the 
Ministry is to recover funds if the ED group 
provides fewer hours of work than determined 
under the Ministry's formula; conversely, the 
Ministry must make an additional payment if 
the ED group provides more hours, whether 
due to the volume or acuity of patients 
increasing. When we reviewed reconciliation 
summaries prepared by the Ministry for the 
five-year period from the 2005/06 fiscal year 
through the 2009/10 fiscal year, we identified 
Ministry overpayments totalling $3.9 million. 
These overpayments were made to 24 ED 
groups, with 10 ED groups receiving overpay- 
ments in more than one year. The Ministry 
indicated that it had chosen not to recover 
the overpayments because the recovery could 
negatively affect patients' access to ED servi- 
ces or increase ED wait times. 



• When we reviewed Ministry payments made 
in the 2009/10 fiscal year (the last full year 
for which data were available at the time of 
our audit) to ED groups funded under the 
24-hour model, we noted that over 35% of the 
ED groups sampled received more funding 
than stipulated in their contract with the Min- 
istry. Excess funding in 2009/10 amounted to 
over $400,000, none of which was recovered 
by the Ministry. The Ministry indicated that 
the ability of these EDs to provide physician 
coverage 24 hours a day, seven days a week, 
would have been compromised without this 
additional funding. 

Northern Specialist Physicians 

Alternate funding arrangements for northern 
specialists were introduced effective April I, 2008. 
For the 2009/10 fiscal year, 280 physicians received 
payments totalling $39 million under the northern 
specialist alternate funding arrangements. This 
included about $5 million primarily related to base 
funding for the prior year. 

The alternate funding arrangements for northern 
specialists were determined as a result of negotia- 
tions with the Ontario Medical Association, similarly 
to other alternate funding arrangements. However, 
unlike negotiations for other funding arrangements, 
special payments were made to northern specialists 
during the negotiation process. Specifically: 

• In May 2007, the Ministry paid $15,000 each 
to 234 physicians, who gave the Ministry and 
Ontario Medical Association permission to 
collect information on the physicians' income 
from universities, hospitals, and the Ministry, 
through fee-for-service billings, for the pur- 
pose of negotiating the northern specialist 
alternate funding arrangement. 

• In April 2008, the Ministry paid over $15 mil- 
lion in total to 292 physicians who signed a 
document indicating that they planned to 
join a northern specialist alternate funding 
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arrangement, effective April 1, 2008. The 
Ministry informed us that this money was 
funding for the previous fiscal year (which 
ended March 31, 2008) and was paid to 
these physicians in addition to their regular 
fee-for-service earnings through OHIP. The 
amount paid ranged from $20,000 to $70,000 
per physician, with most physicians receiving 
$55,000. The document physicians signed 
indicated that the money was to be returned 
to the Ministry if the physician did not join 
a northern specialist alternate funding 
arrangement. We noted that 39 physicians, 
who collectively received over $1.1 million, 
did not join a northern specialist alternate 
funding arrangement. Contrary to the docu- 
ment signed, the Ministry subsequently 
allowed these physicians to keep the money 
as long as they joined any type of alternate 
funding arrangement and continued to 
practise in Northern Ontario. However, 11 
of these physicians did not join any type of 
alternate funding arrangement. The Ministry 
did not recover any of the $617,000 paid to 
these 11 physicians. 



RECOMMENDATION 2 



To better ensure that payments made under 
alternate funding arrangements among similar 
specialist groups are in accordance with the 
underlying contracts, the Ministry of Health and 
Long-Term Care should: 

• simplify the numerous different types of pay- 
ments under the academic contracts; and 

• review situations where additional funding 
is consistently being provided or where 
overfunding or duplicate payments have 
occurred in order to determine whether the 
funding should be adjusted or recovered. 



| MINISTRY RESPONSE 




The Ministry supports this recommendation. 
As the Auditor General has noted, payments to 



specialists under the alternate funding arrange- 
ments are complicated, because there are num- 
erous types of payments and various premiums 
that specialists can earn. The Ministry agrees 
with the Auditor General's observation that 
there is an opportunity to simplify or reduce the 
number of payments, particularly when they are 
similar to or have outlived their necessity to be 
distinguished from base funding. 

The Ministry is also reviewing practices with 
respect to recoveries; however, the Ministry 
notes that there may be cases, such as for the 
emergency department alternate funding 
arrangements, where pursuing a recovery 
could jeopardize the ability of some emergency 
departments to provide services 24 hours a day, 
seven days a week. 



MONITORING ALTERNATE FUNDING 
ARRANGEMENTS 

We reviewed the Ministry's monitoring of alternate 
funding arrangements with academic physicians 
and emergency department physicians. 

The fee-for-service payment method encour- 
ages physicians to see as many patients as possible, 
because they get paid based on services provided. 
However, most alternate funding arrangements 
do not compensate physicians based solely on the 
volume of patient services provided. As a result, it 
is important that alternate funding arrangements 
with physicians be properly monitored to ensure 
that specialists maintain a minimum level of patient 
services. As well, it is important that the Ministry 
track the costs of each alternate funding arrange- 
ment and evaluate whether the alternate funding 
arrangements are meeting the Ministry's health- 
care goals in a cost-effective manner. 

Shadow billing occurs when physicians partici- 
pating in certain alternate funding arrangements 
(for example, academic comprehensive and 
emergency department) submit data to OHIP on 
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patient services provided. These physicians are paid 
a percentage (which varies by alternate funding 
plan) of the established amount that fee-for-service 
physicians receive for providing these services. 
Shadow-billing data can be used to assess the level 
of services provided by specialists participating 
in alternate funding arrangements and is used by 
at least one other Canadian jurisdiction for such 
purposes. However, the Ministry informed us that 
it has not analyzed shadow-billing claims to deter- 
mine the number of patients seen or the clinical 
services provided. 

Further, under the Academic Health Science 
Centre (AHSC) contracts, physicians can bill 
100% of the fee-for-service claim value for clinical 
services provided, on top of the other amounts 
paid in the contract. Similarly, under the northern 
specialist contracts, physicians can bill 70% of the 
fee-for-service claim value, on top of the other 
amounts paid. The Ministry does not track the total 
fee-for-service amounts paid under either of these 
arrangements. Therefore it does not include these 
payments, which we would expect to be significant, 
when it determines the total amounts paid under 
the AHSC and northern specialist arrangements. 
Without this information, the Ministry does not 
know the total amounts paid to physicians under 
these arrangements. In addition, because a con- 
siderable proportion of the payments under the 
AHSC contracts goes to the governing group for 
distribution to the physicians, instead of directly to 
individual physicians, the Ministry does not know 
the total amount of compensation received by each 
physician participating in an AHSC and therefore 
the reasonableness of the amounts cannot be peri- 
odically assessed. 

The Ministry acknowledged that this informa- 
tion would be useful and advised us that the 
Institute for Clinical Evaluative Sciences (ICES) is 
currently performing a review of physician com- 
pensation by specialty and it expects to receive a 
copy of the report from ICES by spring 2012. The 
Ministry also indicated that it commenced a review 
of AHSC funding, including physician-level fund- 



ing information, which it expects to complete in 
December 2011. 

Because patient volume and acuity form the basis 
of funding for emergency department physicians, 
the Ministry obtains information about both from 
emergency departments funded through alternate 
funding arrangements. We found that the Ministry 
made use of this information to identify over- and 
underpayments to emergency departments. As well, 
the Ministry had a process for preventing excess 
fee-for-service billings in certain circumstances, and 
received information on projected staffing shortages 
in emergency departments. 

However, the Ministry's monitoring of the aca- 
demic contracts was not effective. The contracts for 
academic physicians paid through alternate fund- 
ing arrangements require that their governance 
groups submit numerous reports, such as an annual 
business plan, audited financial statements, a finan- 
cial report, and a human resource report. Although 
we found that the Ministry received much of this 
information, it was not reviewing or analyzing it. 

We concluded that the Ministry has little assur- 
ance that specialists provided the service levels 
outlined in their contracts. The Ministry informed 
us that it performs minimal direct monitoring 
as it expects specialists funded under academic 
contracts to meet their contract obligations, such as 
providing minimum hours of service or spending a 
minimum percentage of their time seeing patients. 
For example: 

• There are three academic comprehensive 
arrangements in place, all of which require 
that the physicians in the group collectively 
work a minimum number of full-time hours. 
Two of the physician groups with these alter- 
nate funding arrangements submit reports to 
the Ministry that contain information that can 
be used to verify the total number of physician 
hours worked. The third hospital did not sub- 
mit such information and the Ministry had not 
followed up to request the information. 

• Under the academic comprehensive contracts, 
specialists are also required to provide a 
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minimum level of clinical patient services. 
For example, specialists working under one 
academic comprehensive contract are required 
to allocate 75% of their time to clinical services 
and the remainder to teaching, research, and 
administrative activities. However, at the time 
of our audit, the Ministry informed us that it 
was not obtaining information on whether 
the physicians were allocating 75% of their 
time to clinical services because this was 
considered a guideline, not a requirement. In 
another example, specialists working under 
another academic comprehensive agreement 
are required to provide a minimum of 33 hours 
of clinical services per week. Although the 
Ministry received some information annually 
on how physicians spent their time at work, it 
did not receive any information on how many 
hours of clinical service the physicians actually 
provided. The Ministry indicated that it does 
not set the hours of work for these physicians, 
and therefore it is considering the use of other 
service-level indicators, such as those in the 
AHSC contracts, in the future. 
Specialists working under AHSC contracts 
are required to provide a minimum level of 
clinical services, including seeing a minimum 
number of patients. The AHSC agreements 
state that if the physicians' services fall below 
an established level for each specialty, the Min- 
istry may reduce the specialists' funding for 
that year. In April 2010, the Ministry reviewed 
information on the service levels achieved by 
the AHSCs for the 2007/08 and 2008/09 fiscal 
years and found that although over 60% of the 
AHSCs had met service-level requirements in 
all their specialty areas, the remaining 40% 
generally had at least one specialty area that 
did not meet the service-level requirements 
specified under the contract. No adjustment to 
future funding levels was made, nor was any 
funding recovered by the Ministry. 
The Ministry promotes self -monitoring for 
physicians participating in an AHSC arrange- 



ment. In July 2010, the Ministry sent letters 
to the governance groups of the AHSC alter- 
nate funding arrangements asking them to 
perform a self-assessment using a checklist 
provided. This checklist was developed by 
the Ministry to help AHSCs assess whether 
they are meeting their obligations under their 
alternate funding agreements. The checklist 
covered areas such as governance, provision 
of services, and reporting requirements. 
The checklist asked whether processes were 
in place to monitor whether direct patient 
services and on-call services were provided. 
However, there was no actual requirement for 
the AHSCs to complete the self-assessments 
or to return completed self-assessments to the 
Ministry. The Ministry indicated that in future 
years it would be requesting confirmation that 
the AHSCs had completed the assessment. To 
ensure that the AHSCs take appropriate action 
on issues noted in the checklist, the Ministry is 
also considering whether or not to request the 
results of the assessment in the future. 
We also noted that the Ministry does not period- 
ically review whether its overall goals and objectives 
for specialist alternate funding arrangements — such 
as improving patient access; supporting the clinical 
training needs of medical students, physicians, and 
other health-care providers; and advancing innova- 
tion in medicine — are being met. 



RECOMMENDATION 3 



To better ensure that Ontarians have access to 
specialist physician care, consistent with the 
overall objective of alternate funding arrange- 
ments, the Ministry of Health and Long-Term 
Care should monitor whether specialist groups 
are providing patient care and other services in 
accordance with their contracts. 

Further, to ensure that the benefits of the 
specialist alternate funding arrangements out- 
weigh the costs, the Ministry should track the 
full costs of each alternate funding arrangement, 
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including total fee-for-service billings paid to 
physicians, either directly or indirectly and use 
this information to periodically review whether 
its overall goals and objectives for such arrange- 
ments are being met in a cost-effective manner. 



The Ministry supports this recommendation 
and the inclusion of appropriate performance 
measures in all alternate funding arrangements. 
Further, the Ministry agrees that the special- 
ist arrangements must be clear and detailed 
and that they must be actively monitored and 
reviewed in order to ensure that physician 
groups are providing patient care and other ser- 
vices in accordance with their contracts. Work 



is under way to develop regular reporting of all 
physician payments under each agreement. An 
enhanced internal monitoring process has been 
developed and will be implemented in the near 
future. This will allow the Ministry to undertake 
regular reviews of the clinical services provided 
by specialists and to undertake periodic costing 
analyses. 

The Ministry is currently evaluating spe- 
cialist payments under the Academic Health 
Science Centre and academic comprehensive 
arrangements for the 2009/10 and 2010/11 fis- 
cal years, which will also enable the Ministry to 
assess the reported level of clinical services and 
academic activities for both physician groups 
and individual physicians. 
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Background 



The Liquor Control Board of Ontario (LCBO) is a 
Crown agency incorporated under the Liquor Con- 
trol Act (Act) with the power to buy, import, distrib- 
ute, and sell beverage alcohol products in Ontario. 
It reports to the Minister of Finance. 

The LCBO's mission is to be a socially respon- 
sible, performance-driven, innovative, and profit- 
able retailer. Overall, it offers consumers more than 
21,000 products — approximately 3,400 items on its 
general list, 6,200 Vintages products, and 12,000 
products available by private order. Vintages, the 
LCBO's fine wine and premium spirits line of busi- 
ness, contributes about 8% of total sales. About 10% 
of general-list products and about 50% of Vintages 
products were newly acquired for the 2009/10 fiscal 
year. The LCBO operates five warehouses that sup- 
ply more than 600 stores across the province. 

The LCBO uses three methods to select and buy 
new products. The principal one, for both the gen- 
eral list and Vintages, is to issue a call to suppliers, 
known as a "needs letter," for a specific category of 
product. It can also purchase products on an "ad 
hoc" basis or, in the case of Vintages products, buy 
directly from suppliers. 

For the 2010/11 fiscal year, the LCBO's sales and 
other income were approximately $4.6 billion, and 



net income was $1.56 billion, with the LCBO remit- 
ting virtually all of that profit to the province. LCBO 
sales have increased by 67% compared to 10 years 
ago, and its net income and the dividends it pays to 
the province have gone up by about 80% over the 
same period. The breakdown of sales by product 
category for the 2010/11 fiscal year in dollars and 
by percentage of contribution to gross margin is 
illustrated in Figure 1. 

Figure 1: Total Beverage Alcohol Sales and 
Contribution to Gross Margin by Category, 2010/11 

Source of data: Liquor Control Board of Ontario 
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35 


35 


beer 


0.91 


20 
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4.54 
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Audit Objective and Scope 



The objective of our audit was to assess whether 
the LCBO had adequate systems, policies, and pro- 
cedures in place with respect to new product pur- 
chases, and whether such purchases were acquired 
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and managed effectively and in compliance with 
applicable legislation, government directives, and 
LCBO procurement policies. 

Before we began our audit fieldwork, we identi- 
fied the audit criteria we would use to address our 
audit objective, then designed and conducted tests 
and procedures to address them. Our audit object- 
ive and criteria were reviewed and agreed to by 
LCBO senior management. 

In conducting our audit, we reviewed relevant 
legislation and administrative policies and pro- 
cedures, and we interviewed appropriate LCBO 
head-office staff and, where applicable, Ministry of 
Finance staff. We also reviewed and assessed per- 
tinent summary information and statistics, as well 
as a sample of general-list and Vintages purchasing 
files. We held discussions with several key beverage 
alcohol stakeholder groups, including Spirits Can- 
ada, Drinks Ontario, the Wine Council of Ontario, 
and the Winery and Grower Alliance of Ontario. 
These groups represent both large and small bev- 
erage alcohol suppliers and agents. In addition, 
we met with a representative from the Canadian 
Association of Liquor Jurisdictions, whose members 
include the liquor jurisdictions of the 13 provinces 
and territories in Canada. We also contacted the 
liquor organizations in British Columbia, Alberta, 
and Quebec to get an understanding of their oper- 
ations for comparison purposes, and the National 
Alcohol Beverage Control Association in the United 
States, which represents 18 states that directly 
control the distribution and sale of certain types of 
beverage alcohol in their jurisdictions. 

Our audit included a review of related activities 
of the LCBO's Internal Audit Services. We reviewed 
its recent reports and considered its audit work and 
any relevant issues it had identified when planning 
our work. 




Under Ontario's Liquor Control Act (Act), the Liquor 
Control Board of Ontario (LCBO) has the power 
to set the retail prices for the beverage alcohol 
products it sells. The mandate it follows in doing so 
is to promote social responsibility in the sale and 
consumption of alcohol while generating revenue 
for the province. In this regard, the Act sets out 
minimum retail prices for alcohol — certain other 
jurisdictions, such as Quebec and New York State, 
have no such minimums. This means that the 
LCBO does not sell its products at the lowest prices 
possible but rather at levels aimed at encouraging 
responsible consumption and generating profit for 
the government. 

Most Canadian jurisdictions operate in a similar 
context. Canadian beverage alcohol products gen- 
erally have higher markups and taxes as compared 
to alcohol sold in the United States, which means 
that many products are sold at lower prices in the 
U.S. Although some of the products that the LCBO 
sells are offered at lower prices in other Canadian 
jurisdictions, an April 2011 survey found that the 
LCBO had the lowest overall beverage alcohol retail 
prices of all Canadian liquor jurisdictions, with the 
third-lowest prices for spirits and beer, and the low- 
est wine prices. In a 2010 LCBO customer survey, 
about two-thirds of the respondents agreed that the 
LCBO's prices represented good value. 

The LCBO's purchasing process differs from 
those used by private-sector retailers. In the private 
sector, retailers attempt to buy their products at the 
lowest possible cost. Although one might expect the 
LCBO — one of the largest purchasers of alcohol in 
the world — to follow a similar approach, the LCBO's 
purchasing process does not focus on cost. Instead, 
it focuses on the retail price it wants to charge for 
a product. Suppliers submit a retail price within an 
established retail price range set out in the LCBO's 
call for products and then work backwards, apply- 
ing the LCBO's fixed-pricing structure to determine 



2011 Annual Report of the Office of the Auditor General of Ontario 



the wholesale cost they will charge the LCBO. If a 
supplier's cost quote results in an amount that does 
not match the agreed-upon retail price, the LCBO 
will ask it to raise or lower the wholesale cost of the 
product. We found examples both where the suppli- 
er's initial cost quote had been raised and where it 
had been lowered, usually by fairly small amounts. 
But we also found examples where suppliers sub- 
mitted wholesale quotes that were significantly 
lower or higher than what the LCBO expected, so 
the LCBO requested that the supplier revise the 
quote, which effectively either raised or lowered 
the price it paid the supplier for the product. The 
LCBO informed us that such discrepancies between 
the submitted quote and the expected amount may 
have been due to changes in transportation costs, 
currency exchange rates, or a supplier's error in 
calculating its quote. 

Most large retailers use their buying power to 
negotiate with suppliers to drive down costs. We 
found that the LCBO does not negotiate discounts 
for high-volume purchases to reduce its costs. This 
is also true of the other Canadian jurisdictions we 
looked at. The LCBO's fixed-pricing structure gives 
it no incentive to negotiate lower wholesale costs — 
doing so would result in lower retail prices and, 
in turn, lower profits, which the LCBO indicated 
would be contrary to its mandate of generating 
profits for the province and encouraging respon- 
sible consumption. The LCBO has been successful 
in consistently generating increased profits for the 
province year after year. 

The LCBO has many well-established purchasing 
practices that are consistent with those in other 
Canadian jurisdictions and in other government 
monopolies, such as in Sweden and Norway. How- 
ever, the LCBO could improve some of its processes 
relating to purchasing and subsequent monitoring 
of product performance to better demonstrate that 
these are carried out in a fair and transparent man- 
ner. For example: 

• In a sample that we examined, the LCBO 
rejected more than 80% of shortlisted 
products under the needs-letter process but 



documented the reasons for very few of its 
decisions. 

• The LCBO did not have written policies 
and procedures to help its staff determine 
under what circumstances it is appropriate 
to purchase products on an ad hoc or direct 
basis, and what evaluation criteria should be 
used, although 60% of all new products in 
the 2010/11 fiscal year were purchased using 
these two procurement methods. 

• There was no documented rationale for the 
60% of the products in our sample purchased 
using the ad hoc process. However, LCBO staff 
generally were able to recall the rationales 
behind their purchasing decisions when we 
discussed these with them. 

• With respect to direct purchases of Vintages 
products, more than 45% of the direct pur- 
chases in our sample had not been previously 
purchased by the LCBO, and therefore never 
underwent the required taste, sight, and 
smell assessment. In addition, there was no 
documentation of the reasons for purchasing 
them. For those that were purchased directly 
because the LCBO had purchased them in the 
past, our review of historical sales noted that 
the amount of sales for most was less than the 
required sales threshold, and that many of 
these same products continued to sell poorly. 

• The LCBO has a well-defined category 
management process and sets sales targets 
by product category to assess product per- 
formance and deal with products that are 
performing badly. It can delist products that 
fail to meet sales goals and request rebates. 
However, we found that some products that 
failed to meet sales goals for four or five years 
had not been delisted, and we also noted that 
the LCBO requested and received rebates for 
only 35% of delisted products. 
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I OVERALL LCBO RESPONSI 



The LCBO appreciates the Auditor General's 
observations that the LCBO's purchasing 
practices are well established and similar to the 
practices of other liquor boards in Canada and 
abroad. The Auditor General's recommenda- 
tions for enhancing these practices will help 
the LCBO further improve current practices; 
many of the recommendations are already being 
implemented. 

LCBO buyers make every effort to get the 
best products in every price band, whether for 
wines under $8 or those greater than $100. 
They review more than 50,000 submissions 
annually and work with suppliers to make the 
best of these available at good prices. The LCBO 
is an attractive customer for manufacturers, 
and there is fierce competition for listings. As a 
result, suppliers frequently submit products to 
the LCBO at prices lower than those at which 
they sell for in other Canadian jurisdictions. It is 
a mandatory condition of sales to the LCBO that 
suppliers do not sell the same product at a lower 
price to other Canadian liquor retailers. Regular 
price surveys show that the LCBO has the lowest 
overall alcohol prices in Canada. 

The audit examined new product procure- 
ment, but suppliers of products already listed 
can unilaterally lower their retail price by selling 
to the LCBO at a reduced cost. Again, strong 
competition between listed products causes this 
to happen. 

The LCBO must achieve a balance among the 
elements of its mandate: generating revenue, 
promoting social responsibility, and providing 
customers with selection and value at all price 
points. Although the audit report states that 
"the LCBO does not sell its products at the low- 
est prices possible," in fact many products in the 
LCBO do sell for the lowest price legally allow- 
able. Under its mandate and the fixed-markup 
structure, the LCBO cannot currently negotiate 
for volume discounts. The National Alcohol 



Beverage Control Association confirms that 
although there is variation at the state level, 
wholesale discounts are not permitted in most 
U.S. states. Similarly, as the audit report notes, 
other Canadian liquor boards also do not negoti- 
ate volume discounts. In light of this, and in 
addition to obtaining the best products at very 
competitive prices, the LCBO also obtains more 
than $100 million annually from suppliers to 
support promotions and in-store merchandising. 



Detailed Audit Observations 



RETAIL PRICES OF BEVERAGE ALCOHOL 
PRODUCTS 

Under the Liquor Control Act (Act), the LCBO has 
the power to set the retail prices for beverage alco- 
hol in Ontario. It sets the prices for spirits, imported 
wines, and most Ontario wines, as well as for the 
beers that it sells exclusively. Beer manufacturers 
set the retail prices for other beers, in accordance 
with the Act. 

In setting retail prices, the LCBO considers key 
aspects of its overall mandate and objectives to 
generate sufficient profits for the provincial gov- 
ernment, promote social responsibility in the sale 
and consumption of beverage alcohol, support the 
province's wine industry, and offer customers good 
product selection and value at all price points. 

A number of federal and provincial taxes, fees, 
and levies must be incorporated into LCBO prices, 
as illustrated in Figure 2. However, the two com- 
ponents that account for most of the retail price of 
alcohol are the wholesale price (described more 
fully in the later section titled "The Cost of Beverage 
Alcohol Products") and the LCBO's markup. 

The markup structure was established by the 
LCBO and approved by its Board of Directors in con- 
sultation with the Ministry of Finance (Ministry) . 
The LCBO advised us that successive governments 
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Figure 2: Components of Retail Prices for Alcohol Products 

Source of data: Liquor Control Board of Ontario 


Component 


Explanation 


supplier quote 


price at which the supplier sells its product to the LCBO 


+ federal excise tax 


a variable tax based on volume and alcohol content 


+ federal import duty 


applied only to imported goods; similar in structure to excise tax 


+ freight 


rate based on existing LCBO freight contracts 


= landed cost 


+ cost of service charge 


applied to beer* products at a fixed rate per litre 


+ LCBO markup 


variable rate applied according to the type of product (see Figure 3) 


+ wine levy 


applied to most wine products at a fixed rate per litre 


+ bottle levy 


applied to all products at a fixed rate per litre 


+ environmental charge 


applied to all non-refillable products at a fixed rate per container 


= base price 


+ harmonized sales tax (HST) 


fixed rate of 13% applied to the basic price 


+ container deposit 


amount varies depending on volume of container (up to $0.20) 


= final retail price 


prices are rounded up to the next $0.05 



* The LCBO sets the price of beer products that are exclusively sold by it. In accordance with the Liquor Control Act, retail prices for beer products that are also 
sold by The Beer Store are set by the manufacturer. 



have endorsed the fixed-markup structure, and 
thus a fixed-pricing structure, as an appropriate 
means for a government agency such as the LCBO 
to show transparency in its purchasing and pricing 
practices. Markup rates vary by product category. 
However, within each category, the markup is 
the same for all products (examples are shown 
in Figure 3) . Although the markup applies to all 
products, the LCBO can, at its discretion and with 
Board approval, change the markup on Vintages 
products. In addition, international trade agree- 
ments require government agencies such as the 
LCBO to treat domestic and imported products in 
the same way by applying the same markups to 
similar domestic and imported products. However, 
the LCBO can apply additional charges on imported 
products to cover any extra costs associated with 
such purchases. 

The pricing structure and the related impact of 
LCBO's mandate and its objectives are well under- 
stood by suppliers and agents. Although some of this 
information is disclosed on the LCBO website under 
the "Contact Us" section, it would not be easy for 



the public to find this information or use it to fully 
understand how beverage alcohol prices are set. 

Retail Price Comparisons 

The LCBO conducts retail price comparisons 
across Canada on a quarterly basis on behalf of 
the Canadian Association of Liquor Jurisdictions 
(Association), whose members include all 13 liquor 
jurisdictions in Canada. Prices are compared for 
some 50 products from different categories (except 
Vintages) that generate high sales revenue across 
Canada and that are sold in all 13 jurisdictions. The 
most recent survey results at the time of our audit 
showed that the LCBO had the third-lowest retail 
prices for spirits and beer, and the lowest for wine. 
The LCBO also had the lowest overall beverage 
alcohol retail prices. 

We selected a sample of popular LCBO products 
to compare their retail prices to those in British Col- 
umbia, Alberta, and Quebec, as well as New York, 
Michigan, and Pennsylvania. Prices varied across, 
and sometimes within, these jurisdictions, but we 
noted that: 
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Figure 3: Examples of Markup Rates for Spirits, Wines, 
and Beer 

Source of data: Liquor Control Board of Ontario 





Markup Applied 


1 Class of Products 


Product Origin 


% ($/litre) 


vodka, whisky, rum 


domestic 


141.0 




imported (U.S.) 


148.1 




imported (other) 


148.0 


table wine 


Ontario 


65.5 




other domestic 
and imported 


71.5 


beer 


domestic and 
imported 


.7094 




microbrewers 


.2095 



• Two other Canadian provinces sold almost 
all of the products at higher prices. The third 
province sold many of its products at prices 
lower than the LCBO, but all of its prices were 
within 7% of the LCBO's prices. This result 
was generally consistent with the Associa- 
tion's survey findings discussed at the begin- 
ning of this section. 

• The three states sold most of the products at 
prices 3% to 55% lower than the LCBO. 

The LCBO does not sell products at the lowest 
price possible because doing so would be contrary 
to its social responsibility mandate and would 
reduce its profits. For example, Ontario has min- 
imum retail prices set out by the Act, while Quebec, 
Alberta, New York, and Pennsylvania have no such 
minimum prices. According to the LCBO, Canadian 
jurisdictions have consistently higher markups and 
taxes than those in the U.S. to encourage socially 
responsible consumption while generating revenue 
for government. The LCBO also pointed out that its 
2010 customer survey found that about two-thirds 
of respondents agreed that it charged prices that 
represented good value. 

Legislated Minimum Retail Prices 

The government has established minimum retail 
prices for alcohol products to moderate consump- 



tion. Public health research shows that higher 
alcohol prices lead to a reduction in drinking and 
the consequences of alcohol use and abuse. Ontario 
is one of six Canadian jurisdictions that have min- 
imum retail prices. 

With senior management approval, retail prices 
for discontinued products and products that will 
soon become unsellable because of age are allowed 
to fall below these minimum thresholds. However, 
such exceptions are not allowed for beer. 

Minimum pricing applies to products sold in all 
Ontario stores, including LCBO retail stores and 
non-LCBO stores (such as agency stores in com- 
munities with populations too small to support a 
regular LCBO store, The Beer Store, and manufac- 
turers' stores such as winery and distillery retail- 
ers) . Minimum retail prices for beer and spirits are 
established by legislation. For wine, the legislation 
establishes minimum prices that the LCBO must 
pay its suppliers. According to the LCBO, the deci- 
sion to set a minimum cost for wines was made in 
the 1980s to prevent the dumping of foreign wines 
into Ontario at subsidized prices. For the purpose of 
this report, we have converted the minimum acqui- 
sition cost for wine into a minimum retail price by 
applying the LCBO's fixed-pricing structure, as illus- 
trated in Figure 4. Both the minimum retail price 
and the minimum acquisition cost are adjusted for 
inflation on March 1 of each year in accordance 
with the Act. 

We were informed by staff at the Ministry that 
the LCBO is responsible for ensuring that all stores 

Figure 4: Examples of Minimum Retail Prices as of 
March 1, 2011 ($) 

Source of data: Liquor Control Act, Liquor Control Board of Ontario 





Minimum Retail Price 


1 Product Type 


(excluding bottle deposit) 


spirits - 750 mL 


23.20 


table wine, Ontario - 750 mi- 


5.65 


table wine, imported - 750 mL 


5.80 


beer* 


26.40 



* case of 24 341-mL bottles with alcohol content equal to or greater than 
4.9% and less than 5.6% 
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selling alcohol in Ontario, including non-LCBO 
stores, comply with the minimum pricing require- 
ments. However, LCBO staff said they were unclear 
about their role in enforcing minimum prices, espe- 
cially considering that some non-LCBO stores could 
be viewed as competition. The LCBO also noted 
that the Act does not specify the mechanisms by 
which these requirements are to be enforced. This 
increases the potential for products to be sold below 
their minimum retail price in contravention of the 
LCBO's social responsibility mandate. 



RECOMMENDATION 1 



To better inform Ontarians about how beverage 
alcohol prices are set, the LCBO should provide 
more information to the public on its pricing 
policy, including how its mandate and provin- 
cial policy objectives affect pricing, and details 
about its pricing structure. As well, the LCBO, 
in conjunction with the Ministry of Finance, 
should establish a process for ensuring that all 
stores are complying with the Liquor Control 
Act's minimum retail price requirements and 
consider whether the LCBO is the most appro- 
priate organization to monitor this compliance. 




The LCBO agrees that Ontarians should have 
easy access to information about its mandate 
and operations, including pricing policy. That 
is why this information is currently available 
on the agency's website (www.lcbo.com). It is 
also routinely shared with the beverage alcohol 
trade and the media. The LCBO will examine 
ways to make it easier to locate this information 
and present it in the clearest way possible. 

The LCBO will work with the Ministry of 
Finance to review how compliance with min- 
imum prices is assured in all beverage alcohol 
retail locations across the province and which 
agency or part of government should fulfill this 
regulatory function. 



The Cost of Beverage Alcohol Products 

In the private sector, retailers attempt to buy their 
products at the lowest possible cost. One might 
also expect the LCBO, as one of the world's largest 
purchasers of beverage alcohol, to use its buying 
power to negotiate the lowest possible costs from its 
suppliers, but the LCBO's purchasing process does 
not focus on cost. Instead, the LCBO focuses on set- 
ting a specific retail price range, and suppliers must 
offer their products to the LCBO within this range. 
The wholesale price, or cost of the product to the 
LCBO, is determined by applying the LCBO's fixed- 
pricing structure and working backwards from the 
supplier's retail price to arrive at the cost the sup- 
plier will be paid for its product. 

The LCBO's purchasing process is consistent 
with those in other Canadian jurisdictions we 
contacted and also resembles the ones used by 
other government monopolies, such as Sweden 
and Norway. The LCBO also informed us that, in 
keeping with its mandate to encourage responsible 
drinking, its retail pricing is geared toward 
"premiumization." This strategy aims to generate 
more revenue without increasing the amount of 
alcohol that is consumed by encouraging consum- 
ers to buy more premium-priced products and by 
increasing the number and variety of products 
offered at higher prices. 

Once the LCBO determines the category of 
products it wishes to purchase, it issues a "needs 
letter," which is posted on its website and sent to 
various trade organizations. A needs letter outlines 
the required product category, the product specifica- 
tions, and, in almost all cases, the retail price ranges. 
The retail price ranges that the LCBO asks for vary 
depending on the type of product requested and are 
based on what the LCBO establishes as prices that 
the market will bear. Most needs letters establish a 
price range, but some establish a firm price that sup- 
pliers must meet or exceed. For instance, the annual 
needs letter for general-list wines issued for the 
2010/11 fiscal year requested 48 product categor- 
ies. Of these, 41 required the retail price to exceed 
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$9.95, while another six had to exceed $8.95 (one 
did not stipulate a price range) . The LCBO advised 
us that most of the requested wines were required to 
retail for more than $9.95 because it already had an 
adequate selection of wines under $10. 

The retail price at which a supplier wants its 
product to sell in Ontario must be within the price 
range established in the LCBO's needs letter. As 
noted earlier, the wholesale cost to the LCBO is 
determined by applying the fixed-pricing structure 
and working backwards from the agreed-upon 
retail price. In essence, the LCBO sets the cost it will 
pay for a product by first establishing its retail price. 
We also found that the LCBO does not negotiate 
discounts for high-volume purchases to reduce its 
costs, but neither do the other Canadian jurisdic- 
tions we looked at. With its fixed-pricing structure, 
the LCBO has no incentive to reduce the supplier's 
wholesale cost, because doing so would result in 
lower retail prices and, in turn, lower profits — 
which would be contrary to the LCBO's mandate. 

After the LCBO has selected a product it wants 
to purchase, the supplier sends a written confirma- 
tion or quote for the product. The LCBO inputs 
the supplier's quote into its system and applies 
the fixed-pricing formula. The result should equal 
the retail price in the supplier's submission. If the 
calculation results in an amount greater than the 
agreed-upon retail price, the LCBO will request that 
the supplier lower its wholesale quote to ensure 
that the LCBO is not overcharged. However, if the 
wholesale quote results in a lower retail price than 
what has been agreed to, the LCBO does not accept 
the quote, because the application of its fixed- 
pricing structure would result in a retail price that 
is too low. In such cases the LCBO asks the supplier 
to increase the quote, effectively raising the price it 
will pay for the product. 

Although we found in a sample of products that 
some supplier quotes had been raised while others 
had been lowered, for the most part by minor 
amounts, we did note that while some suppliers 
submitted higher wholesale quotes, others submit- 
ted quotes that were significantly lower than what 



had been expected for some of the products in our 
sample. As a result, these suppliers were requested 
to revise their wholesale price. The LCBO informed 
us that differences in quotes may have been due to 
changes in transportation costs, currency exchange 
rates, or a supplier's error in calculating its quote in 
order to match the required retail price. However, 
the reasons for adjusting the quotes were not 
documented. 

The process just described did not always apply 
to products acquired for Vintages. We found that 
suppliers of these products were generally not 
asked to adjust their quotes. Instead, the LCBO 
adjusted its markup to arrive at the agreed-upon 
retail price. 

We also noted that, as a condition to its purchase 
agreement, the LCBO requires suppliers to agree 
that they will not sell their product to the LCBO at 
a higher wholesale price than the price at which it 
is sold to any other government liquor purchaser in 
Canada, in the same quantities. If this happens, the 
LCBO is entitled to a rebate for the difference. We 
found that, for general-list products, the LCBO did 
not regularly monitor and compare its supplier costs 
to those paid by other jurisdictions for the same 
products. LCBO management indicated that it was 
difficult to obtain supplier cost information from 
other jurisdictions. Reasons for this included con- 
cerns that other jurisdictions considered the terms 
of payment to their suppliers confidential informa- 
tion. The LCBO also noted that other Canadian 
jurisdictions may purchase products at a lower cost 
than the LCBO, but that it would be unable to pur- 
chase them at this wholesale price if the application 
of its fixed-pricing structure resulted in a retail 
price lower than that allowed by the Act. 

For Vintages products, the LCBO informed us it 
occasionally scans the Internet and reviews trade 
papers to identify retail price differences. In one 
case, it identified a supplier that charged the LCBO 
more than another Canadian jurisdiction for three 
Vintages products from 2006 to 2009. The LCBO 
requested and received rebates from this supplier 
totalling approximately $600,000. 
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RECOMMENDATION 2 



In keeping with its mandate to generate suf- 
ficient profits and adhere to the government's 
policy direction of maintaining a retail pricing 
mechanism that encourages responsible 
consumption, the LCBO should consider, in 
consultation with the Ministry of Finance, the 
following strategy on a trial or pilot basis to take 
advantage of its being one of the largest purchas- 
ers of beverage alcohol products in the world: 

• once product categories and their related 
retail price ranges have been determined, 
allow suppliers to offer a product at what- 
ever cost they are willing to accept to have 
it sold at the LCBO, and then use a variable 
markup to arrive at the desired fixed retail 
price; and 

• calculate the gross profit margin for a par- 
ticular product based on the supplier's cost 
quote, and take this into consideration in 
making decisions on which new products to 
purchase along with the other evaluation 
criteria currently used, such as the quality of 
the product. 



| LCBO RESPONSE 




Previous independent reviews of the LCBO, 
commissioned by the province, have examined 
variable or flexible markups. The current fixed- 
markup structure, and the LCBO's use and man- 
agement of it, has been endorsed by successive 
governments as appropriate for a government 
agency such as the LCBO. The current structure 
provides certainty and transparency for suppliers 
and more easily ensures that foreign products 
are treated as trade agreements require. 

The LCBO looks forward to discussing the 
recommendation of a pilot of variable markups 
with the Ministry of Finance and will support 
the government's policy analysis of this pro- 
posed change. 



As the Auditor General noted, the LCBO's 
fixed-markup structure results in product needs 
letters identifying retail price ranges for new 
products rather than a supplier quote or the 
payment the LCBO makes for the product to the 
supplier. LCBO buyers and suppliers or their 
agents focus on retail price, and once the price 
has been established, the supplier quote follows 
from the mechanical application of the fixed- 
pricing structure. 

The LCBO is a single buyer for a major mar- 
ket. As a result, suppliers are keen to have their 
products listed for sale in LCBO stores. Com- 
petition for listings is fierce, and suppliers and 
their agents strive to submit the best products 
they have in response to LCBO product calls. By 
leveraging its size and buying power, the LCBO 
is able to obtain the best products the world has 
to offer at prices lower than those in comparable 
jurisdictions. When a product call asks for wines 
between $13 and $15, for example, suppliers 
often choose to offer wines that they would 
prefer to retail for $16 to $18 because of the 
attractiveness of large-volume orders from the 
LCBO. This is a central way that the LCBO meets 
one of the key objectives of its mandate: offering 
its customers good product selection and value 
at all price points. 



IDENTIFYING PRODUCT NEEDS 

The LCBO conducts a planning process to identify 
the types of products it will buy for the forthcoming 
year. The process includes establishing a budget 
that encompasses sales targets and the targeted 
amount of the dividend it will pay to the province. 
The LCBO provides this information to the Ministry 
and incorporates the Ministry's feedback before 
finalizing its targets. 

An overall business plan sets out the LCBO's 
financial goals, such as total sales, litres to be sold, 
and average gross margin, and its non-financial 
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objectives, such as development of a varied product 
assortment to meet customer needs. These are 
further divided into high-level business plans for 
Vintages, beer and spirits, and wines. These include 
sales and gross-margin targets for each category as 
well as some detail on where to focus the forthcom- 
ing year's purchases. The LCBO uses a well-defined 
category management system for product purchas- 
ing. Annual category reviews identify growing 
or declining categories and gaps in the product 
selection where there may not be products available 
at certain retail price ranges, and this information 
is included in detailed category plans. These plans 
are also based on research to identify key trends 
and customer preferences. Prior sales histories also 
help staff to determine which products to purchase. 
Most respondents to the LCBO's 2010 customer 
survey said it offered a wide product selection, and 
more than half said there were often new products 
available to purchase. 

Once these business plans are developed, the 
LCBO posts the needs letters described in the previ- 
ous section on its website and distributes them 
to various trade associations. Needs letters are 
generally issued once during the fiscal year for each 
general-list product category, and three to four times 
a year for Vintages products. Needs letters include a 
number of requests for purchases. For example, the 
2011 general-list needs letters included requests for 
purchases of 68 different types of products. 

We selected a sample of the LCBO's needs letters 
to determine whether the products requested were 
consistent with those identified in the applicable 
fiscal year's detailed business plans. We found that 
purchases of spirits and beer in the 2009/10 fiscal 
year and Vintages purchases for both 2009/10 and 
2010/11 were consistent with the business plans 
for those years. However, we noted that no detailed 
category plans had been created for the entire 
general-list category in 2010/11 or for the wine 
category in 2009/10. High-level business plans had 
been developed in 2011 and draft versions were 
available for 2010, but these provided staff with 
only limited direction. The LCBO informed us that 



detailed plans had not been created for these years 
because of staff turnover and restructuring. Accord- 
ing to the LCBO, an evaluation was conducted 
using information such as the historical sales 
performance of each product, trend analysis, price- 
band analysis, and an assessment of the inventory 
levels and overall performance of the portfolio. 
However, this evaluation was not documented and, 
as a result, it was not apparent what direction was 
provided to staff to guide their purchasing and 
whether the purchases they made were consistent 
with the direction they received. 



RECOMMENDATION 3 



To help ensure that purchases reflect corporate 
sales objectives and meet customer demand, the 
LCBO should develop detailed annual category 
plans for the major beverage alcohol categories. 



LCBO RESPONSE 



The two LCBO business units and Vintages 
produce business plans every fiscal year. These 
business plans provide the context to produce 
detailed category management plans. The LCBO 
will ensure that these plans are produced for the 
major categories every year. 



METHODS OF PURCHASING NEW 
BEVERAGE ALCOHOL PRODUCTS 

The Management Board of Cabinet's Procurement 
Directive (Directive) establishes comprehensive 
policies for the procurement of all goods by min- 
istries and government agencies. Although the 
procurement of alcohol products is subjective in 
nature, the principles in the Directive are still fun- 
damental to procurement across the public sector. 
These principles include open access for qualified 
vendors; conducting the procurement process in a 
fair and transparent manner; procuring goods only 
after considering the business requirements; and 
the effective management of procurement through 
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the appropriate policies and procedures. The Direc- 
tive also sets out guiding principles that must be 
followed in the evaluation process. These include 
the evaluation of bid responses in a consistent man- 
ner and in accordance with the evaluation criteria, 
rating, and methodology set out in the procurement 
document. They also require the evaluation process 
to be fully disclosed, including a clear statement 
of all mandatory requirements, all weightings for 
rated requirements, and a description of any short- 
listing process. Following the evaluation process, 
only the highest-ranked submissions that have met 
all mandatory requirements may be selected. 

According to its memorandum of understanding 
with the Ministry, the LCBO must comply with the 
Directive. It may develop and use its own internal 
policies so long as these policies incorporate the 
principles set out in the Directive. Because the 
procurement of alcohol is a specialized area, the 
LCBO chose to develop its own policy. We found 
that its policy for general-list products, with the 
exception of a description of the prequalification 
or shortlisting process, was based on the principles 
established in the Directive. However, at the time 
of our audit the LCBO had no documented policies 
related to Vintages' purchasing, although Vintages 
was in the process of creating such documentation. 

The LCBO purchases new products throughout 
the year using three different methods: requests 
for purchase through the needs-letter process, pur- 
chases on an ad hoc basis, or purchases direct from 
suppliers. The number and percentage of purchases 
by each purchasing method for the 2010/11 fiscal 
year are shown in Figure 5. 



Purchases by Needs Letters 

As noted previously, needs letters setting out details 
including the countries, regions, varietals, and 
retail price ranges of products that the LCBO wants 
to purchase are posted on the LCBO's website and 
sent to various trade organizations. As shown in 
Figure 5, most general-list purchases are made via 
needs letters. Suppliers send their product submis- 
sions electronically, including such information as 
supplier name, product name, product price, a one- 
page marketing plan, relevant product attributes, 
third-party accolades, and photos of the product 
and packaging. 

Because supplier responses to needs letters 
can number in the hundreds or even thousands, 
the LCBO shortlists a more manageable number 
of qualified responses before proceeding to the 
next stage. For general-list products, the use of a 
prequalification or shortlist stage started in the 
summer of 2010; Vintages had already been using 
a prequalification stage for a number of years. The 
LCBO informed us that, because it does not have a 
specific policy for the prequalification stage, a num- 
ber of informal factors are considered when assess- 
ing submissions, including whether the product 
received a high score from a third-party reviewer, 
the types of awards it has won, whether it is a high- 
demand product with limited availability, and the 
product's image and packaging. 

We reviewed a sample of needs letters issued 
in the 2009/10 and 2010/11 fiscal years for both 
general-list and Vintages products. We noted that 
more than 80% of the submissions in our sample 



Figure 5: Types of Purchases by Purchasing Method, 2010/11 

Source of data: Liquor Control Board of Ontario 
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were declined during this prequalification stage, 
but there was documentation to support the deci- 
sion to decline for only a few of them. There was 
documentation to support the decision to shortlist 
the product for only a few of the remaining 20% of 
accepted submissions. 

We also noted that approximately 10% of the 
accepted submissions that were shortlisted for a 
more detailed assessment did not fall within the 
retail price range established in the LCBO's needs 
letter. For instance, one Vintages needs letter speci- 
fied a range of $14 to $50, yet one of the accepted 
submissions had a retail price of $64. One needs 
letter for general-list wines requested a price range 
of $11.95 to $16.95, and five of the accepted sub- 
missions quoted a price of $9.95. The rationale for 
accepting these products was not documented. 

We were informed that submissions outside of 
the needs specified are generally rejected, but excep- 
tions may be made at the discretion of the LCBO if 
the product is deemed to be outstanding and to meet 
the LCBO's financial and product-variety objectives. 

According to the LCBO, management oversees 
the product selection process. However, we did not 
note any documented approval for these shortlisted 
products, including those products selected even 
though they did not meet all the requirements set 
out in the needs letter. The LCBO's Internal Audit 
Services also noted in its July 2010 review of the 
selection process for new products that the LCBO 
should document its basis for selecting products that 
were not within the established needs-letter criteria. 

The next stage in the assessment includes an 
organoleptic (taste, sight, and smell) evaluation, 
and a label and container assessment. Suppliers 
send a sample of the product for organoleptic assess- 
ment by a panel of LCBO employees, including those 
who work in the category's purchasing unit and 
specially trained store employees. For some Vintages 
products, an employee who holds a Master of Wines 
designation is also included in the panel. A chemical 
analysis is also performed on the product, generally 
in the LCBO's own laboratory, to ensure that it is 
safe to drink and contains its stated contents. 



For general-list products, suppliers must submit 
a detailed plan, which includes information such 
as sales from previous purchases by the LCBO, 
sales generated in other provinces, and marketing 
information. Suppliers of general-list products are 
informed that their submissions will be evaluated 
on a scale of 100 points, assigned as follows: sales 
information (10); marketing information (45); 
organoleptic assessment (20); and sample package 
appeal (25) . The cost of the product is not a factor 
in the selection process because the needs letter 
has already specified what the retail price (and 
hence what the cost to LCBO) will be. Suppliers 
of Vintages products are not required to submit a 
detailed plan; rather, we were informed that the 
tasting evaluation was the key determinant. Other 
factors taken into consideration include third-party 
reviews, price/ quality ratio, and availability of 
inventory. The entire process, from the issuing of 
the needs letter to the selection of the product, 
takes approximately three and a half months. 

We reviewed a sample of general-list and Vin- 
tages product submissions and noted considerable 
variations in the level of documentation to support 
the decisions made. For example: 

• In some cases there was no documented 
rationale for a purchase, while in others the 
reasons for accepting or declining the product 
were clear. 

• In many cases, the submission was not scored, 
and there was no evidence that the marketing 
and sales information was evaluated, even 
though these components accounted for 55% 
of the overall assessment. However, each 
panel member's organoleptic assessments and 
packaging and price/ quality ratio assessments 
were documented for all general-list products. 
LCBO management indicated that the scoring 
structure was outdated and did not reflect the 
differing priorities of each product category, 
but there was no evidence that another scoring 
system had been formally established and used 
to assess these submissions. 
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• Although organoleptic evaluation is the key 
criterion for Vintages products, the LCBO 
was unable to locate the organoleptic scores 
for 143 of the Vintages products in our 
sample, although it ended up purchasing 
60 of these. The organoleptic scores for the 
rest of the items in our Vintages sample were 
documented for each panel member. We were 
informed that other factors, such as third- 
party reviews or historical sales, are taken into 
consideration during the purchasing process, 
but that these factors are not formally scored. 
Overall, because the LCBO does not rank or 
summarize its assessment for each product submis- 
sion, it was not evident why certain products were 
selected for purchase. We also noted that when 
submissions were declined, the supplier was sent 
an email notifying it that "other products submitted 
performed better against selection criteria." We 
were informed that suppliers may contact the LCBO 
to obtain more detailed information on why their 
product was declined. However, given the lack of 
documentation, it would be difficult in our view for 
LCBO staff to subsequently provide suppliers with 
more specific details. 

In its July 2010 report, the LCBO's Internal Audit 
Services noted similar concerns and recommended 
that staff be given guidelines to help support their 
decisions for purchasing new products. Such guide- 
lines would include the establishment of product 
selection criteria and an evaluation that would be 
subject to annual management review and approval. 

Ad Hoc Purchases 

When warranted, the LCBO can purchase products 
on an ad hoc basis, outside of the needs-letter pro- 
cess. The situations in which this may be necessary 
include shifts in consumer purchasing behaviour 
for products not currently available at the LCBO, 
needs-letter calls that were not timely enough to 
address a shift in consumer purchasing behaviour, 
or the identification of new or innovative products 
with favourable accolades. Ad hoc purchases may 



be initiated by LCBO staff, or suppliers may inform 
the LCBO about a specific product they feel the 
LCBO might want to purchase. We were informed 
that very few products are purchased directly from 
supplier-initiated offers. 

A product must be approved by LCBO manage- 
ment for ad hoc purchase before it is accepted, and 
then it is subject to taste-testing and the rest of the 
regular procurement process. In the 2010/11 fiscal 
year, 15% of spirits and 28% of both wine and beer 
purchases were made on an ad hoc basis, as shown 
in Figure 5. 

At the time of our audit, the LCBO did not 
have written policies and procedures to guide its 
ad hoc purchasing process. Staff told us that they 
applied informal criteria, such as whether a product 
met a current gap in the product category, or the 
innovativeness of a product. We were informed that 
all decisions to accept an ad hoc submission were 
reviewed with management prior to finalizing the 
decision, but this review was not documented in 
most instances. The LCBO's Internal Audit Services 
also noted a need for formalized ad hoc purchasing 
policies in its July 2010 review. 

We reviewed a sample of general-list and Vinta- 
ges ad hoc purchases and noted that for 60% of the 
products purchased there was no documentation 
explaining why the product had been purchased on 
this basis. However, when we asked about certain 
products, LCBO staff were generally able to provide a 
verbal rationale to support these purchase decisions. 

Direct Vintages Purchases 

Direct purchases are made either during the 
prequalification stage of the needs-letter purchas- 
ing process or from products submitted directly by 
the supplier with prior LCBO permission. Only Vin- 
tages products can be procured by direct purchase 
and, as noted in Figure 5, about 60% of all Vintages 
products purchased in the 2010/2011 fiscal year 
were acquired in this way. Products that the LCBO 
decides to purchase directly do not require organo- 
leptic evaluation. 
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At the time of our audit, the LCBO did not have 
written policies and procedures in place to guide 
the direct purchasing process. Therefore, no formal- 
ized evaluation criteria existed to assess product 
submissions and there was no guidance on the level 
and type of documentation required and the type of 
management approval needed to support purchas- 
ing decisions. The LCBO informed us that the main 
considerations in the selection of products for direct 
purchase included high historical sales of previ- 
ously purchased products, defined as sales of more 
than 75% of product inventory within 12 weeks; a 
compelling case showing that the product is in high 
demand in other markets and therefore likely will 
sell well in Ontario; or high third-party accolades. 
The LCBO also noted that it selects products for dir- 
ect purchase when its capacity to perform additional 
organoleptic assessments is limited. 

We reviewed a sample of supplier submissions 
in response to the LCBO's needs letters from the 
2009/10 and 2010/11 fiscal years and found that 
there was often no documentation showing the 
rationale for making a direct purchase from them. 
Specifically: 

• More than 45% of the direct purchases in our 
sample were for products that had not been 
previously purchased by the LCBO and there- 
fore had never had an organoleptic assess- 
ment. No reasons for these direct purchases 
were documented. 

• The remaining products selected for direct 
purchase had been bought in the past by the 
LCBO. About half of these purchases had 
no documentation of the buyer's review of 
the related historical sales of the product or 
other support for purchasing them. For those 
that did have documentation of the prior 
sales history, we found that more than 50% 
of these products had sales of less than the 
required 75% of product inventory within 

a 12-week period, yet were still selected for 
direct purchase. For instance, one product had 
previous sales of only 4%, and another just 
37%. According to the LCBO, the previously 



purchased products may have been from a dif- 
ferent vintage, and therefore were not exactly 
the same. Another possible explanation is that 
some new circumstance may have arisen to 
make the product more attractive than its past 
sales might suggest, such as a new review or 
a price change. However, we found no docu- 
mentation showing the reasons these products 
were selected. 
• To determine how well the products selected 
for direct purchase were selling, we reviewed 
sales data for those products that had per- 
formed poorly in the past and compared it to 
subsequent sales data. More than half of those 
products that had performed poorly in the 
past continued to perform poorly. 
Although the LCBO has no formal policy requir- 
ing management approval of significant direct 
purchases, we were informed that such purchases 
may be presented to the category manager for 
validation or verbal approval. However, for the 
direct purchases in our sample, there was no docu- 
mentation of management having approved the 
products selected. 



RECOMMENDATION 4 



To ensure that it can demonstrate to suppli- 
ers and other stakeholders that purchases are 
acquired through an open, fair, and transparent 
process, the LCBO should: 

• develop written policies and procedures for 
each procurement method, including the 
evaluation criteria and process to be used in 
assessing submissions at the various stages 
of the procurement process; 

• disclose its evaluation criteria to suppliers, 
including a clear articulation of all manda- 
tory requirements, an indication of the 
relative weighting for rated requirements 
where applicable, and a description of the 
shortlisting process; and 

• ensure that reasons for selection and 
required management approvals are appro- 
priately documented. 
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The LCBO is in the process of updating its 
Product Management Policies and Procedures 
manual, which will include a description of 
the shortlisting process. The LCBO was in the 
process of documenting its Vintages policies 
during the audit and provided them to the 
Auditor General when the audit fieldwork was 
completed. The LCBO will continue to disclose 
its purchasing policies and procedures, and any 
changes or updates to them, to the beverage 
alcohol trade on its Trade Resources website at 
www.lcbotrade.com. 

The LCBO agrees that the reasons for 
product acceptance and required management 
approvals should be documented. The LCBO 
will continue to record new product accept- 
ance decisions in the New Item Submission 
System and has implemented a new process for 
documenting the rationale for these decisions 
from the submission stage onward, including 
management approvals. The LCBO has added 
further oversight to the shortlisting of product 
applications at the pre-submission stage, where 
the product manager reviews all applications 
with the category manager, whose documented 
approval is now required. 

The LCBO has concerns regarding the idea of 
documenting the reasons for selection or elim- 
ination of products at the pre-submission stage. 
Currently, the LCBO receives more than 50,000 
submissions annually and anticipates that this 
number will continue to grow. Documenting the 
reasons why products are accepted or declined 
at this stage would entail either limiting the 
number of submissions the LCBO would accept 
or hiring additional staff. Neither would result 
in a better product assortment. 

Product submissions are assessed against 
general evaluation criteria, combined with the 
experience and judgment of the LCBO's buyers. 
This is a common retail-sector practice and one 
that the LCBO believes is appropriate for its pur- 



chases. LCBO product purchase risk is further 
mitigated by the use of sales targets or sales per- 
formance terms to ensure that the products per- 
form well in the market. This allows the LCBO 
to responsibly discontinue any underperforming 
products and to continue to create opportunities 
for new products. 



ONGOING MONITORING OF PRODUCT 
PERFORMANCE 

Monitoring the sales performance of products is 
done to help identify, in a timely manner, products 
that are not selling well and that should be replaced 
with better-performing ones. 

The LCBO sets various sales targets by product 
category and adjusts them annually. Sales targets 
for wines, spirits, and Vintages core products 
(called Essentials) are based on the sales in dollars 
achieved in the previous year, while beer targets are 
based on the number of litres sold in the previous 
year. Targets are set to retain 90% to 95% of the 
products. 

The LCBO informed us that the targets did not 
consider factors such as limited store distribution of 
products and niche products, although, according 
to LCBO staff, these factors were considered in their 
reviews of product performance. LCBO manage- 
ment informed us that targets can be adjusted as 
necessary. However, we found that there was no 
policy or guidance on when it is appropriate to 
adjust the sales targets or by how much. 

New products are not held to their targets in 
their first year of release. Monitoring of a product 
occurs on a monthly basis at the start of the second 
year for general-list products and at the start of the 
third year for Vintages Essentials products. The 
LCBO produces a monthly report that tracks, on 
a 13-month rolling average, product sales against 
targets. However, it does not track the length of time 
that a product has failed to meet its sales targets. We 
selected a sample of individual products from the 
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general-list and Vintages Essentials categories that 
had not met their sales targets for approximately 16 
consecutive months as of early 2011 and reviewed 
their performance for the previous four years. We 
found that often, timely action had not been taken 
to address poorly performing products, and that 
some of these products had not been meeting their 
sales targets over a period of four or five years. 

In the event that a product fails to meet its per- 
formance targets, the LCBO may consider delisting 
or discontinuing it. LCBO management told us 
that they work with suppliers to try to increase 
sales — through, for example, additional market- 
ing or promotional campaigns — before delisting a 
product. When a product underperforms and the 
LCBO decides to delist it, it may request a rebate in 
accordance with its purchasing conditions. It asks 
for 25% back on the cost of remaining inventory 
for general-list and Vintages Essentials products, 
and 20% back for Vintages Retail Release products 
(new products released every two weeks for a 
limited time) . The LCBO's policy does not specify 
how long a product may continue to underperform 
before it is delisted and when it is appropriate to 
request a rebate from the supplier. The decision 
to request a rebate is left to the judgment of LCBO 
management. The LCBO informed us that Vintages 
Essentials products that do not meet sales targets 
maybe periodically transferred out of the Essen- 
tials program and offered through other Vintages 
programs, and therefore rebates are generally not 
claimed for these products. However, for other 
types of products, we noted that the LCBO often did 
not ask suppliers to provide rebates when it delisted 
poorly performing products. In particular, we noted 
that 2,270 general-list and Vintages Retail Release 
products were delisted from April 2009 to around 
December 2010, and the LCBO had requested and 
received rebates for about 35% of them. 

Products for which rebates have been requested 
and received are marked down at LCBO stores. 
However, prices remain the same for those where 
no rebates have been sought. The LCBO informed 
us that it uses its judgment on a case-by-case basis 



to determine when to request rebates. For instance, 
it may not ask for a rebate when sales are deemed 
close enough to the target, or if the product comes 
from an iconic supplier with which the LCBO does 
not want to jeopardize its relationship. But there is 
no requirement to document these decisions. 



RECOMMENDATION 5 



To help ensure that products not meeting 
acceptable sales targets are identified in a timely 
manner, the LCBO should: 

• regularly review and assess sales targets for 
each product category to ensure that they 
continue to be reasonable and appropriate 
for identifying underperforming products; 

• establish clear guidelines for the nature and 
timing of action to be taken when a product 
is identified as underperforming; and 

• establish policies for documenting decisions 
on delisting and requesting supplier rebates. 

| LCBO RESPONSE 

LCBO sales targets are reviewed and updated 
annually. In the future, Vintages Essentials 
sales targets will also be reviewed and updated 
annually. 

The LCBO agrees to establish clear guide- 
lines for the nature and timing of action to be 
taken when a product is identified as underper- 
forming and for developing policies to docu- 
ment the decision to delist products and/or seek 
a supplier rebate. 

With respect to the reference to the LCBO 
requesting and receiving rebates on only 35% 
of delisted products, it is important to note that 
rebates are requested only for products that 
are reduced in price to clear inventory. Rebates 
compensate for part of the reduced profit from 
these sales. The LCBO avoids rebates where pos- 
sible in order to maximize revenue. 



Chapter 3 



Ministry of the Attorney General 



Section! 

3.09 Legal Aid Ontario 




Background 




The Legal Aid Services Act, 1998 (Act) took effect 
in April 1999 and established Legal Aid Ontario 
as an independent corporation accountable to the 
Attorney General to provide legal assistance to 
low-income individuals. Between 1967 and 1999, a 
similar function had been performed through the 
Ontario Legal Aid Plan, which was administered by 
the Law Society of Upper Canada (Law Society) . 

Under the Act, Legal Aid Ontario is required to 
provide "consistently high quality legal aid services 
in a cost-effective and efficient manner" to clients 
deemed eligible, and is to encourage and facilitate 
flexibility and innovation in the provision of such 
services, while recognizing the private bar and clin- 
ics as the foundation for providing such services in 
the areas of criminal and family law, and clinic law, 
respectively. 

Legal Aid Ontario provides assistance in three 
ways: 

• It issues more than 100,000 legal aid certifi- 
cates annually to people involved in criminal, 
family, and immigration/refugee law matters, 
and in certain civil matters. Clients apply 
for certificates primarily through Legal Aid 
Ontario's offices in courthouses, and through 
its client call centre. About 99% of clients who 



are able to obtain a certificate retain private 
lawyers, who in turn bill Legal Aid Ontario for 
the services they provide to the client. About 
4,700 private-sector lawyers participate in 
the legal aid program. Legal Aid Ontario also 
operates law offices with staff who provide 
legal services to people who have certificates. 

• It pays and manages about 1,500 staff and 
contract lawyers to provide duty counsel 
services at criminal and family courts. Duty 
counsel primarily provide legal representation 
and advice to eligible people appearing in 
court without counsel. 

• It funds and oversees 77 independent com- 
munity legal clinics, with nearly 550 staff who 
assist low-income people with clinic law issues 
such as government-assistance matters and 
representation at tribunals such as those deal- 
ing with landlord-tenant disputes. Funding is 
also provided to legal aid clinics operating at 
six universities with law programs. 

In Ontario, the income threshold for eligibil- 
ity for legal aid certificates is very low and has 
not changed since 1996. According to Legal Aid 
Ontario, about 80% of approved applicants have 
gross annual incomes under $10,000, and the 
majority are on some form of social assistance or 
have no reported income. 

Legal Aid Ontario employs about 700 staff at 
its head office in Toronto, nine district offices, 
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70 courthouses, and 10 law offices. As shown in 
Figure 1, Legal Aid Ontario received $354 million 
in funding during the 2010/11 fiscal year, with 76% 
of that coming from the provincial government. 
Additional funds came from the federal govern- 
ment under a cost-sharing agreement, from the 
Law Foundation of Ontario, and from clients whose 
income levels require them to help pay for legal 
assistance. As shown in Figure 2, Legal Aid Ontario 
incurred $362 million in operating expenditures, 
with $315 million spent on client programs and 
$47 million on administration and other expenses. 



Audit Objectives and Sco 



Our audit objective was to assess whether Legal 
Aid Ontario had adequate systems, processes, and 
procedures in place to: 

• ensure that consistent high-quality legal aid 
services are delivered in a cost-effective and 
efficient manner to low-income individuals 
throughout Ontario in accordance with legis- 
lated requirements; and 

• measure and report on its effectiveness in 
doing so. 

Figure 1: Revenue by Funding Source, 2010/11 
($ million) 

Source of data: Legal Aid Ontario 

Total Revenue = $354 

Law Foundation of Ontario ($14) other income ($1) 

\ / 

client contributions ($18) 
federal government ($53) 



province ($268) 




Senior management reviewed and agreed to our 
audit objective and associated audit criteria. 

We conducted our fieldwork at Legal Aid 
Ontario's head office in Toronto and visited five dis- 
trict offices, five courthouses, and three law offices. 
Our work included interviewing staff, reviewing 
recent reports and studies, and examining policies, 
records, and systems. We also held interviews with 
representatives of eight community legal clinics, as 
well as criminal, family, and refugee lawyers' asso- 
ciations to discuss their perspectives on legal aid 
services provided in Ontario. We also considered 
recommendations we made in our last audit of 
Legal Aid Ontario in 2001. 

We researched legal aid programs in other juris- 
dictions and met with senior program managers in 
two other provinces. We also engaged an independ- 
ent expert who has senior management experience 
in delivering public legal aid programs. 

Legal Aid Ontario's internal auditor, and 
consultants that it hired, conducted a number of 
recent reviews that were helpful in planning our 
audit, including an assessment of the adequacy, 
effectiveness, and efficiency of internal controls 
over its lawyer billing and payment system, reviews 
of about half of the community legal clinics, and a 
value-for-money review of duty counsel services at 
two court locations. 

Figure 2: Program Expenditures, 2010/11 ($ million) 

Source of data: Legal Aid Ontario 



Total Expenditures = $362 



administration and service 
provider support ($47) 



law offices and other 
client support ($25) 



certificates ($178) 



clinics funding ($68) 



duty counsel ($44) 
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more clients have been required to rely on duty 
counsel, legal advice, and information from 
Legal Aid Ontario's website for legal services. 
• Although the Act requires Legal Aid Ontario 
to establish a quality assurance program with 
the Law Society with respect to the work 
performed by lawyers, it has not implemented 
quality assurance audits of lawyers since Legal 
Aid Ontario's inception in 1999. A robust qual- 
ity assurance program would help ensure that 
legal services provided by staff and contract 



lawyers to low-income and vulnerable clients 
are of a high standard. Given that legal aid 
lawyers generally work for lower rates than 
those in private practice, and approximately 
11% of them carried large caseloads repre- 
senting almost half of all certificates issued, 
there is a need for quality assurance audits to 
ensure that legal services meet the legislative 
requirement that certificate work performed 
by lawyers be of consistently high quality. 

• At the time of our audit, Legal Aid Ontario was 
working on system improvements to address 
deficiencies with its lawyer payment system. 
Most importantly, strengthening of controls 
is required to ensure that all payments, which 
total $188 million annually, are justified. 

• Legal Aid Ontario's efforts to extract greater 
efficiencies from community legal clinics have 
caused relations to deteriorate. Although 
the Act technically gives Legal Aid Ontario 
significant authority and control over all areas 
of clinics' operations and expenditures, this 
has tended to conflict with the clinics' culture 
of independence and their individual board of 
director governance structure. 

• With the significant amount of solicitor-client 
privileged information on its information tech- 
nology systems, we expected Legal Aid Ontario 
to have performed recent and comprehensive 
privacy and threat risk assessments of its 
computer databases. However, the last privacy 
assessment was in 2004, and its systems have 
changed significantly since then. 

As with our 2001 audit, we again noted that Legal 
Aid Ontario lacks key performance measures on the 
services it provides to its clients and stakeholders, 
and its annual reporting was three years overdue. In 
addition, it has not reported publicly on its strategic 
and business plans in a comprehensive manner. 



For at least the last decade, Ontario has spent more 
on legal aid support per capita than any other prov- 
ince, even though it has one of the lowest income 
eligibility thresholds and issues fewer certificates 
entitling people to legal aid per capita than most 
other provinces. Legal Aid Ontario acknowledges 
the need to address a history of operating deficits, 
make its operations more cost-effective, improve 
access to its services, and help make the courts 
more efficient. We noted that it has a well-defined 
long-term strategy to address these issues and that 
it has moved to increase access to legal aid services 
beyond the issuing of certificates, such as through 
expanded use of duty counsel available at court- 
houses and through its new call centre. 

We felt its multi-year long-term strategy was 
heading in the right direction. However, the fol- 
lowing are some of the areas the legal aid program 
must address if it is to be fully successful in meeting 
its mandate: 

• Only people with minimal or no income qual- 
ify for legal aid certificates or for assistance 
from community legal clinics — the financial 
eligibility cut-offs for qualifying have not 
changed since 1996 and 1993, respectively. 
This, combined with an escalation in the aver- 
age legal billing for each certificate issued, 
has meant fewer people over the last couple of 
years have been provided with certificates and 
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H RESPONSE 





Legal Aid Ontario agrees with all of the recom- 
mendations of the Auditor General and wel- 
comes his observation that Legal Aid Ontario's 
Modernization Strategy is heading in the right 
direction. 

As an independent public institution com- 
mitted to working with its clients, the Ministry 
of the Attorney General, and all justice-system 
stakeholders, Legal Aid Ontario looks forward 
to continually improving the legal aid system for 
low-income Ontarians. 



RECENT INITIATIVES 



servations 



In recent years, Legal Aid Ontario has implemented 
significant changes to its operations to control 
costs and improve accessibility for its clients. Many 
of these changes were ongoing at the time of our 
audit. The need for change had been identified 
in public reviews of Legal Aid Ontario, during its 
participation in initiatives by the Ministry of the 
Attorney General (Ministry) to make criminal and 
family courts more efficient, and internally by Legal 
Aid Ontario management as part of its efforts to 
improve efficiency and to address ongoing operat- 
ing deficits. 

A July 2008 report resulting from a review 
commissioned by the Attorney General identified a 
number of themes that have since driven many of 
the changes at Legal Aid Ontario. The report said 
that legal aid reform must be viewed as an integral 
part of a broader justice-system reform, particularly 
with regard to timely resolution of disputes; that 
the income level under which someone must fall to 
qualify for legal aid services must be raised so that 
it bears a better relationship to the actual circum- 
stances of those in need; and that varying levels 



of services should be provided on a sliding scale of 
eligibility. Additionally, it recommended that fees 
paid to lawyers (called "tariff rates") and salaries 
paid to staff lawyers should be increased; that ser- 
vices should be provided in an integrated fashion, 
including more one-stop shopping opportunities for 
clients; that more public electronic information and 
hotline services should be available; and that Legal 
Aid Ontario should receive more funding. 

Other reviews have said that Legal Aid Ontario 
needs to pay tariff rates high enough to attract 
more experienced lawyers to take on the large, 
complex criminal cases it is sometimes called upon 
to handle. Legal Aid Ontario spends an average of 
$20 million per year, or 25% of its criminal cer- 
tificate expenditure, on big cases, but these cases 
represent only 2% of the number of criminal certifi- 
cates issued. In the 2009/10 fiscal year, the average 
cost of a case that was not one of these big cases 
was $1,391; the average big-case cost was $24,700, 
or 18 times the cost of a regular certificate. At the 
time of these reviews, criminal lawyers working on 
legal aid cases were organizing a protest of the Min- 
istry's approved legal aid tariff rates and boycotting 
certain cases. 

The provincial government announced a fund- 
ing increase to Legal Aid Ontario of $51 million 
spread over three years, beginning in the 2007/08 
fiscal year, to improve access for low-income Ontar- 
ians. Legal Aid Ontario officials said that their plan 
for these funds would include a 5% increase to the 
tariff paid to lawyers, exemption of the universal 
child-care benefits from legal aid applicants' 
income, development of a new financial eligibil- 
ity test for applicants, increasing the number of 
certificates for family law matters, an increase in 
funding to the big-case management program, and 
initiatives at community legal clinics to increase 
employee salaries and improve services. 

In September 2009, the provincial government 
also announced a transformation plan for Legal Aid 
Ontario, with additional funding of $150 million 
over four years, including an annual increase in 
base funding growing to $60 million a year by the 
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fourth year. The objectives included expanding 
clinic legal services, developing a faster and easier 
system for the resolution of family law matters, 
promoting the Ministry's Justice on Target project 
for addressing criminal court delays, and creating a 
big-case management office. 

Under the Law Society Act, the Law Founda- 
tion of Ontario pays Legal Aid Ontario 75% of the 
interest earned on deposits held in trust by lawyers 
and paralegals involving transactions such as real- 
estate purchases. Due to the economic downturn 
and declining interest rates, recent revenues from 
the Law Foundation to Legal Aid Ontario had 
decreased sharply, from a peak of $56.4 million 
in 2007/08 to $4.8 million in 2009/10, a decline 
of more than 90%. This decline has largely offset 
the annual funding increases recently announced 
by the government. As a result, over the three 
fiscal years from 2008/09 to 2010/11, Legal Aid 
Ontario incurred operating deficits of $19.1 mil- 
lion, $27.7 million, and $8.6 million, respectively, 
as it tried to manage with less overall revenue than 
expected and higher operating costs than prior to 
this period. 

Legal Aid Ontario management has introduced 
a number of initiatives — including its Value Agenda 
in 2007 and its Modernization Strategy in 2009 — to 
reduce program costs and improve services and 
efficiency. The strategies include plans to reduce 
the use of costly traditional legal services, primarily 
via certificates, which provide for individual rep- 
resentation, and instead make use of new technolo- 
gies and alternative service models for individuals 
whose legal matters do not warrant the issuing 
of a certificate, such as providing duty counsel 
services, summary legal advice, and information 
from its new call centre and an enhanced website. 
A simplified financial eligibility test was established 
to reduce the administrative cost and time it takes 
to review and accept an application. The initiatives 
also include a goal to reduce administrative costs by 
5% over five years, in part by reducing the number 
of district offices to nine from 51; restructuring 
payments and payment procedures for lawyers; 



increasing to 55 from eight the number of legal aid 
offices at courthouses to process legal aid applica- 
tions; and establishing a process to boost the role 
and effectiveness of community legal clinics. 

The Act requires Legal Aid Ontario to submit an 
annual report to the Attorney General within four 
months of its March 31 year-end, and to include key 
information on its activities and results. When the 
Attorney General tables this report in the Legisla- 
tive Assembly, it becomes available for public 
review. However, at the time of our audit in August 
2011, the most recent Legal Aid Ontario annual 
report published was for the year ending March 31, 
2008. Although that report included fairly compre- 
hensive data regarding certificates issued by area 
of law and their cost, and the numbers of assists to 
clients provided by duty counsel and legal clinics, 
the report did not include measures on the quality 
or effectiveness of these programs or of client ser- 
vice in general. 

Two Canadian provinces with large legal aid 
plans that we reviewed issue strategic plans to 
inform the public of their key priorities for the fol- 
lowing five years, and they report annually on the 
progress. For instance, Manitoba outlines its areas 
of strategic focus and its action plan, and identifies 
steps to be taken, who is responsible, implementa- 
tion date, costs, and outcomes and measures. Its 
annual report includes achievements, and the most 
recent report focuses on actions in its strategic 
plan to improve service delivery to clients, improve 
internal support services to its staff, and implement 
a new governance structure. 

In a recent Legal Aid Ontario survey of certifi- 
cate lawyers, 31% of respondents stated that they 
did not have a clear understanding of Legal Aid 
Ontario's strategic direction. Our discussions with 
stakeholders, including lawyers and clinic staff, 
also confirmed that this was a concern. 

As well, although Legal Aid Ontario published 
several documents on its website describing its 
plans for its Modernization Strategy and there 
was some information in its 2008 annual report, 
considering all the changes and initiatives it has 
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undertaken over the last few years, it needs to do 
more to inform the public about these changes, 
how it is executing them, and whether they are 
producing the desired results. 

Furthermore, in its most recent publicly avail- 
able annual report, for 2007/08, Legal Aid Ontario 
stated its intention to develop a plan to establish 
more comprehensive performance measure 
reporting across the organization over three to four 
years. At the time of our audit, Legal Aid Ontario 
was still working on developing performance 
measures to assess how it was making a difference 
to clients and their communities. However, we are 
concerned about the length of time that it is taking 
to complete this work. 



RECOMMENDATION 1 



To better inform the Legislature and the public 
of its strategic priorities and success in achiev- 
ing its mandate of providing legal assistance to 
low-income Ontarians, Legal Aid Ontario should 
develop and implement meaningful perform- 
ance measures on its key services and program 
outcomes, and enhance both the information 
in its annual report and on its website. It should 
also work with the Ministry of the Attorney 
General to ensure that its annual report is made 
public on a more timely basis. 

| LEGAL AID ONTARIO RESPONSE 

Legal Aid Ontario agrees with the Auditor 
General's recommendation. Legal Aid Ontario 
recognizes the importance of communicating 
its areas of strategic focus and action plans with 
the public and stakeholders. Legal Aid Ontario 
maintains a proactive communications program, 
and changes were recently made to the organiza- 
tional structure to enhance Legal Aid Ontario's 
focus on policy, research, and outreach. Legal 
Aid Ontario commits to the development of 
further stakeholder and public communications 
initiatives and policies, and will seek to update 



the memorandum of understanding between the 
Ministry and Legal Aid Ontario in this regard. 

Legal Aid Ontario agrees with the Auditor 
General that meaningful performance measures 
are important. Legal Aid Ontario has developed 
measures to track progress against its Mod- 
ernization Strategy. For example, productivity 
target savings of 1% per year over three years 
were developed and achieved. Legal Aid 
Ontario has developed performance indicators 
that have been included in Legal Aid Ontario's 
reporting to the Ministry and is in the process 
of developing further measures focusing on 
program outcomes. 

Subsequent to the review by the Auditor 
General, Legal Aid Ontario's 2008/09 annual 
report was tabled in the Legislature and is now 
posted on our website. Legal Aid Ontario also 
submitted its 2009/10 annual report to the Min- 
istry of the Attorney General for tabling in the 
Legislature, and its 2010/11 annual report will 
be submitted shortly. 



MEETING DEMAND FOR LEGAL AID 

Under the Act, Legal Aid Ontario's mandate is to 
promote access to justice for low-income people by 
"identifying, assessing and recognizing the diverse 
legal needs of low-income individuals and of 
disadvantaged communities in Ontario," and to do 
so within its available financial resources. Details 
about such matters as applications for legal aid, 
appeals of eligibility decisions, recovery of legal aid 
costs, functions of duty counsel, and appointment 
of lawyer panels that provide services are set out in 
regulations, as are details of recording and billing 
requirements for lawyers, along with their fee and 
tariff schedules. 

Moreover, the ability of Legal Aid Ontario to 
address the needs of its clients is greatly affected 
by the same problems that have hindered the effi- 
ciency of the court system in general. The problems 



2011 Annual Report of the Office of the Auditor General of Ontario 



include the fact that the number of criminal charges 
and family cases handled by the courts has steadily 
increased over the past 10 years. Various factors 
have contributed to backlogs and delays, and courts 
are dealing with longer-running and more complex 
cases. Ministry initiatives such as Justice on Target 
have produced some progress in the last year in 
addressing court backlogs and delays. 

Applicants are eligible for legal aid provided 
they meet the prescribed financial eligibility 
requirements and their legal issue is covered by 
the program. The financial eligibility test considers 
gross income, family size, and assets. Applicants 
below a set income threshold are eligible for a free 
legal aid certificate or other services, while those 
above the threshold up to a predefined limit can 
receive a legal aid certificate if they agree to pay 
for some or all of the services under a contribution 
agreement. The benefits of such agreements are 
that the legal services are provided at what is usu- 
ally a lower legal aid tariff rate and that clients can 
repay Legal Aid Ontario over time. 

As previously noted, the financial eligibility 
threshold for certificates has not changed since 
1996, and only those individuals with little or no 
income qualify. In our sample, only 8% of certificate 
clients were employed with some income, 37% 
were recipients under Ontario Works or the Ontario 
Disability Support Program, and 55% reported no 
income whatsoever. Of those reporting no income, 
more than half were incarcerated. Similarly, Legal 
Aid Ontario reports that 80% of approved appli- 
cants have gross incomes under $10,000, 73% 
either receive social assistance or have no reported 
income, and 94% of certificates were not subject to 
a contribution agreement. 

In 2001, Legal Aid Ontario issued almost 
118,000 certificates at an average cost of almost 
$1,350. In our 2001 audit, we concluded that Legal 
Aid Ontario had not been effective in control- 
ling the costs of its certificates. However, since 
that time, Legal Aid Ontario has been working to 
address these costs. In the 2010/11 fiscal year, legal 
aid certificates accounted for approximately half 



of all Legal Aid Ontario expenditures. As shown in 
Figure 3, just over 100,000 certificates were issued 
and the average cost per certificate had risen to 
$1,752, an increase of about 30% over the 10-year 
period. The most recent spike in cost, from 2009/10 
to 2010/11, is largely attributable to the fact that 
certificates are more frequently being reserved for 
complex and expensive cases, and to increases in 
the tariff rates paid to lawyers, which was done to 
address the rate conflict with criminal lawyers. 

A goal of Legal Aid Ontario's Modernization 
Strategy is to reduce reliance on the use of cer- 
tificates when other, less costly, assistance can be 
provided. Most people without legal representa- 
tion can receive legal aid assistance with criminal, 
family, or refugee/immigration matters from either 
duty counsel or the call centre. However, the fact 
that Legal Aid Ontario's financial eligibility cut-off 
has not changed since 1996 is one of the reasons 
that, given the effects of inflation, fewer people are 
qualifying for certificates. A single person applying 
for a certificate must have an income of less than 
$10,800 a year — an amount so low that someone 
working full time at the minimum hourly wage 
would earn twice as much. 

Figure 3: Legal Aid Certificates Issued and Average 
Cost per Certificate, 2006/07-2010/11 

Source of data: Legal Aid Ontario 
civil 
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As shown in Figure 4, Legal Aid Ontario's eligi- 
bility threshold for a fully paid certificate for a sin- 
gle person is the second-most restrictive among the 
larger provinces. The same is true for the threshold 
requiring a person to repay Legal Aid Ontario under 
a contribution agreement. 

Even though Legal Aid Ontario has been work- 
ing to control its certificate costs, Ontario still 
spends more on legal aid support on a per capita 
basis than any other province, but it is among the 
provinces that issue the lowest number of certifi- 
cates on a per capita basis, as illustrated in Figure 5. 
However, Legal Aid Ontario offers almost three 
times more duty counsel assists per capita than 
the provincial average. Ontario offers more than 
1 million duty counsel assists per year at an average 
cost of $61 each, and makes such assistance avail- 
able to people with higher incomes. A court assist 
is a cost-effective strategy, especially if it can help 
resolve a legal matter without a certificate. In addi- 
tion, independent community legal clinics that are 
almost entirely funded by Legal Aid Ontario assist 
more than 155,000 people with clinic law issues. 

Based on our discussions with various stake- 
holders, we feel that Legal Aid Ontario's multi-year 
reform strategy is heading in the right direction 
since it strives to improve efficiency in service deliv- 
ery and make at least some level of service available 



to a larger number of people, while reserving more 
costly legal representation certificates for more 
serious and complex cases. This approach is consist- 
ent with its legislated mandate, recent studies of 
legal aid, and other reforms to make courts more 
efficient. However, Legal Aid Ontario continues 
to have more costly programs when compared to 
other provinces, which can generally provide cer- 
tificates to more low-income persons because they 
use higher financial eligibility thresholds. 

Because Legal Aid Ontario's Modernization 
Strategy is being deployed to provide less costly 
alternative legal aid services using duty counsel, 
summary legal advice, and information provided 
via its call centre and website, it will be important 
for it to undertake a formal risk assessment to see 
how this approach affects the rights of low-income 
people to legal representation, and to ensure that 
low-income individuals receive the appropriate 
level of legal aid services for their circumstances. 
This risk assessment should take into consideration 
a variety of factors, such as a person's having the 
capacity and knowledge to manage his or her own 
case in the court system, language barriers, mental- 
health issues, and computer literacy. We did note, 
however, that Legal Aid Ontario is taking action to 
address these risks. 



Figure 4: Provincial Comparison of Financial Eligibility for Legal Aid Certificates, as of September 2011 

Prepared by the Office of trie Auditor General of Ontario with data from provincial legal aid offices 





Maximum Income Eligibility Threshold* 




Free Certificate 


With Contribution Agreement 


1 Province 


Family Size: 1 


Family Size: 5 


Family Size: 1 


Family Size: 5 


Saskatchewan 


9,924 


20,784 


12,540 


24,264 


Ontario 


10,800 


26,714 


12,500 


33,960 


Nova Scotia 


12,804 


25,872 


n/a 


n/a 


Quebec 


13,007 


21,328 


18,535 


30,393 


Manitoba 


14,000 


31,000 


23,000 


37,000 


Alberta 


n/a 


n/a 


18,036 


42,312 


British Columbia 


19,632 


59,028 


n/a 


n/a 



*Some eligibility figures have been adjusted for comparison purposes to take into account net income vs. gross income. In addition, most provinces, including 
Ontario, require an applicant's assets to be considered when determining eligibility. We excluded asset assessments from our comparison. Provinces marked 
n/a above: Alberta expects clients to repay some or all of their legal aid bills, but determines eligibility and client contributions on a case-by-case basis; British 
Columbia does not use contribution agreements. 
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Figure 5: Provincial Comparison of Total Legal Aid Funding, 1 Certificates, and Duty Counsel Assists, 2009/10 

Source of data: Statistics Canada 







Total Funding 


Approved 


Duty Counsel 






for Legal Aid 


Certificates per 


Assists per 


1 Province 2 




per Capita ($) 


1,000 Population 


1,000 Population 


Ontario 


28.40 


10 


87 


Manitoba 


26.00 


22 


32 


Nova Scotia 


23.00 


21 


20 


Newfoundland and Labrador 


21.30 


10 


22 


Saskatchewan 


21.00 


21 


16 


Alberta 


20.90 


10 


44 


British Columbia 


17.80 


6 


27 


Quebec 3 


17.30 


29 





Provincial Average 


22.00 


16 


31 



1. Comparison of total legal aid funding does not take into account program differences among provinces, such as areas in law covered, services 
provided, and financial eligibility. 

2. Prince Edward Island and New Brunswick did not report data to Statistics Canada. 

3. Quebec does not provide duty counsel services. 



RECOMMENDATION 2 



To help ensure that its multi-year efforts to 
modernize legal aid services result in delivering 
cost-effective services to those in need, Legal 
Aid Ontario, in collaboration with the Ministry 
of the Attorney General, should: 

• study the impact on low-income individuals 
of its current financial eligibility threshold, 
which has not been raised since 1996, and 
its shift to using less costly legal aid support 
services; 

• assess legal aid programs in other provinces 
to identify the factors and best practices 
contributing to their lower costs that can be 
applied in Ontario; and 

• continue to identify alternative ways to meet 
the legal needs of low-income individuals in 
a cost-effective manner. 

| LEGAL AID ONTARIO RESPONSE 

Legal Aid Ontario agrees with the Auditor 
General's recommendation. Legal Aid Ontario 
has made significant progress and takes care 
to ensure that low-income clients receive cost- 



effective legal aid services tailored to their 
needs while reserving more costly services for 
more complex and serious cases. This allows 
Legal Aid Ontario to serve more clients with its 
available resources. Legal Aid Ontario believes 
the Auditor General's recommendation to study 
the impact of this approach on low-income 
individuals will demonstrate the overall benefits 
of the Modernization Strategy in offering a 
broader range of legal aid services. 

Legal Aid Ontario's financial eligibility 
thresholds are governed by the regulations flow- 
ing from the Legal Aid Services Act, 1998, and 
any changes are the responsibility of the prov- 
incial government. Legal Aid Ontario has been 
concerned about this issue and has discussed 
it with stakeholders and the provincial govern- 
ment over a number of years. Legal Aid Ontario 
will be pleased to assist the Ministry in continu- 
ing to review this important matter. 

Cost comparisons among legal aid plans in 
Canada are difficult to make precisely. It is Legal 
Aid Ontario's opinion that several significant 
limitations apply and that cost per service is 
another meaningful measure. 
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Legal Aid Ontario maintains close contact 
with other provincial and territorial legal aid 
plans through its membership in the Association 
of Legal Aid Plans of Canada, and will continue 
to share information in the areas identified by 
the Auditor General. 



QUALITY OF LEGAL SERVICES 

Panel Management 

In 2004, Legal Aid Ontario began phasing in stan- 
dards that require lawyers to demonstrate a specific 
level of knowledge, skill, and experience in the area 
of law they practice. Those who meet the require- 
ments are assigned to one or more of 10 panels to 
provide service in specific areas of law: criminal; 
extremely serious criminal, such as murder and ter- 
rorism; family; child protection; refugee; consent 
and capacity (mental-health related); duty counsel 
criminal court; duty counsel family court; duty 
counsel advice; and Gladue (aboriginal persons) 
court. Legal Aid Ontario offers support for those 
lawyers through research, learning opportunities, 
and mentoring. 

New lawyers or lawyers new to a particular area 
of law who do not meet the experience requirement 
can be conditionally admitted to a panel if they 
agree to meet the minimum experience level within 
24 months. A conditionally approved lawyer must 
attend training and be mentored, as determined by 
the district area director. Conditionally approved 
lawyers are authorized to accept certificates. 

Legal Aid Ontario's district area directors are 
responsible for assessing applications and admitting 
lawyers to panels in their geographic area. In addi- 
tion, district area directors are responsible for ensur- 
ing that panel membership requirements are met, 
for overseeing the correction of non-compliance 
or unsatisfactory performance or conduct, and, if 
necessary, for initiating steps to remove from a panel 
a lawyer who fails to meet applicable standards. 



Legal Aid Ontario does not set a specific number 
of lawyers for each panel, and the total number of 
panel lawyers increased an average of 5% per year 
over the past five years to 4,700. 

We found that the number of conditionally 
approved lawyers on panels has increased over the 
past five years, from an average of 16% in 2007 to 
23% in 2011. More than 800 of these lawyers had 
spent three years or more on conditional status, 
or at least one year beyond the maximum time 
allowed, and 27 of them had been on conditional 
status since 2004. In addition, we were informed 
that the mentoring process had not been evaluated, 
so Legal Aid Ontario does not know whether there 
are enough mentors available and to what extent 
mentoring needs are being met on a province-wide 
basis. 

Since 2007, panel appointees have been 
required to confirm annually that they have met 
the experience and continuous learning require- 
ments. Requirements vary by panel, but all include 
six hours of legal education and completion of a 
minimum number of case files in the previous year 
in the specific panel area of law. For the 2009 cal- 
endar year (the most recent statistics and status on 
self-reporting available to us), almost 1,100 lawyers 
had not reported on their experience and learning 
activities by the deadline of June 2010 as required. 
We were informed that Legal Aid Ontario cannot 
suspend a lawyer from practising law; only the Law 
Society of Upper Canada (Law Society), the body 
that governs members of the bar, can do so. How- 
ever, Legal Aid Ontario can permanently remove 
a lawyer from a panel. From 2006/07 through 
2010/11, four lawyers were removed from panels, 
two of them related to fraud and overbilling and the 
other two for "reasonable cause." 

We are concerned that insufficient oversight of 
panel management could lead to legal aid clients 
not receiving the expected quality of service, and 
that panel appointees may conclude that the stan- 
dards and reporting requirements are not import- 
ant if little is done to enforce them. 
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Quality Assurance 

The Quality Service Office (Office) of Legal Aid 
Ontario works with district offices, clinics, lawyers, 
duty counsel, and other external stakeholders in 
the justice sector to improve the quality of services 
provided to clients. The Office's responsibilities 
include developing and monitoring panel stan- 
dards for lawyers, providing training seminars and 
materials to lawyers, conducting site visits at clin- 
ics, measuring the satisfaction levels of clients and 
service providers, and developing a performance 
measure framework for Legal Aid Ontario. 

The Act requires Legal Aid Ontario to imple- 
ment a quality assurance program to ensure that 
it is providing high-quality legal aid services in a 
cost-effective and efficient manner. The Act also 
states that Legal Aid Ontario may conduct quality 
assurance audits of providers of legal aid services 
but not of lawyers; it must, instead, direct the 
Law Society to perform quality assurance audits 
of lawyers. Although lawyers are required by 
their professional ethics and conduct standards to 
provide quality services, there is a higher risk that 
legal aid services may not be of a consistently high 
quality because fees paid to lawyers are lower than 
the going rate in private practice. As well, legal aid 
clients are typically more vulnerable and may not 
be as aware that the level of service they receive is 
not adequate. 

In our 2001 audit of Legal Aid Ontario, we 
reported on the lack of a quality assurance program 
to assess the legal aid certificates program. During 
our follow-up in 2003, Legal Aid Ontario indicated 
that it had begun discussions with the Law Society 
on the objectives and approaches common to their 
respective quality assurance programs, and had 
identified areas where there could be co-ordination 
of efforts and support of each other's initiatives. 
However, beyond these initial discussions, little 
action has been taken and there was still no agree- 
ment or memorandum of understanding between 
Legal Aid Ontario and the Law Society, nor were 
there any ongoing efforts to pursue one. Further- 



more, no requests have been made by Legal Aid 
Ontario of the Law Society to carry out any quality 
assurance reviews of lawyers, nor does it do any 
quality assurance audits of lawyers on its own. 

In January 2010, Legal Aid Ontario, along with 
the Ministry of the Attorney General, entered into a 
memorandum of understanding with the Criminal 
Lawyers' Association in which it committed to 
developing and establishing revised requirements 
for panel membership. Legal Aid Ontario did not 
enter into a memorandum of understanding with 
other lawyers' associations, but it agreed to consult 
with them in the development of panel require- 
ments, which would include quality assurance and 
practice review audits, including after-case review. 
However, at the time of our audit there had been no 
progress on these consultations. 

Legal Aid Ontario imposes a billing cap of 2,350 
hours per year to try to ensure that lawyers do not 
overbill and that they do not overextend themselves 
with large caseloads. This works out to a 45-hour 
week for all 52 weeks of the year. We noted that 
over the last three years, approximately 11% of 
panel lawyers carried about 48% of all certificates, 
which on average would require them to work 
almost the maximum number of hours each year to 
complete these files. Legal Aid Ontario has a system 
for identifying lawyers who are approaching the 
annual billing cap, and it then informs the lawyer 
that he or she is approaching the limit. A good start- 
ing point for any quality assurance program would 
be to target those lawyers carrying large caseloads. 

Our research on other legal aid jurisdictions 
showed that in the United Kingdom, formal peer 
reviews are carried out by independent assessors 
funded by the Legal Services Commission of Eng- 
land and Wales. The assessors assign one of five 
grades. The lowest rating carries a recommenda- 
tion that the contract between the Legal Services 
Commission and the lawyer or firm be terminated. 
The second-lowest rating requires a reassessment in 
six months. This approach might be worth investi- 
gating for implementation in Ontario. 
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RECOMMENDATION 3 



To strengthen its ability to ensure that consist- 
ently high-quality legal aid services are being 
provided as required by legislation, Legal Aid 
Ontario should: 

• assess the reasons for a high number of 
lawyers being on conditional status for panel 
membership beyond the two-year maximum 
time allowed, and take timely action to 
ensure that those not meeting requirements 
are appropriately followed up on; and 

• either address long-standing impediments 
to establishing a quality assurance audit 
program with the Law Society of Upper 
Canada or seek changes to its legislation that 
would allow alternative means of developing 
and implementing a quality assurance audit 
program to oversee lawyers, including con- 
sidering best practices in other jurisdictions. 

LEGAL AID ONTARIO RESPONSE 

Legal Aid Ontario agrees with the Auditor Gen- 
eral's recommendation. 

Legal Aid Ontario is committed to ensuring 
that high-quality legal aid services are provided 
by lawyers and to reviewing its process related 
to the conditional status of lawyers on the panel. 

Legal Aid Ontario will develop proposals 
for improving its quality assurance pro- 
gram consistent with the Auditor General's 
recommendations. 



BILLINGS BY LAWYERS 

Payment Systems and Structures 

All private-sector lawyers who accept legal aid 
certificates, as well as court duty counsel who are 
paid on a per diem basis, submit their accounts for 
payment through a Web-based billing and payment 
system. The system was implemented in 2005 to 
process payments more efficiently and help com- 



pensate lawyers in a timely manner. The system 
was updated in 2007 to allow lawyers to accept 
and confirm certificates for clients on-line. In the 
2010/11 fiscal year, the system was modified again 
to accommodate block-fee billing for criminal cer- 
tificates, whereby a fixed fee would be paid for the 
most common legal procedures handled by criminal 
lawyers. Legal Aid Ontario says that it expects the 
new block-fee billing to achieve cost control, reduce 
financial risk, and be easier to administer. 

In 2010/11, the lawyer billing and payment sys- 
tem setded 215,000 billings, totalling $188 million 
in certificate and per diem duty counsel lawyers' 
payments. 

In April 2010, with the assistance of consult- 
ants, Legal Aid Ontario determined that automated 
controls within the billing system did not adequately 
support established billing rules and policies. The 
review also identified $17.5 million in lawyers' 
billings over the previous three-year period that 
warranted follow-up. Among the problems identi- 
fied were possible violations of rules that state that 
lawyers should not bill while under suspension, 
double bill, or bill for unreasonable discretionary 
costs, or work more than 10 hours per day. The 
review also found that there were insufficient 
oversight mechanisms and appropriately trained 
staff in place to ensure that lawyers complied with 
the rules. In addition, lawyers were not required to 
submit court dockets that included the details that 
Legal Aid Ontario staff needed in order to verify the 
work that was billed. A total of 21 recommendations 
were made to address strategic and organizational 
alignment, staff skills and capabilities, operations, 
and technology. 

At the time of our audit, implementation of 
eight recommendations had been completed, and 
implementation of another 12 was expected within 
the following three to six months. We were advised 
at the end of our fieldwork that the system changes 
needed in order to address the remaining recom- 
mendation — to improve the accuracy, timeliness, 
and completeness of the lawyers' database — were 
in progress. 
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Legal Aid Ontario suspended post-payment 
examinations of billings from April 2010 to 
March 2011 because a new targeted, risk-based 
process to examine lawyer account billings was 
being implemented. Billing payments for this per- 
iod amounted to $179 million. Accounts that were 
not examined from April 2010 to March 2011 will 
be subjected to the new examination process within 
the new compliance and risk management frame- 
work. However, we believe that certain payments 
prior to April 2010, including the $17.5 million in 
questionable billings, should be included in the 
examination of past accounts. 

Billing Oversight and Verification 

Legal Aid Ontario's Investigations Department has 
six full-time staff and is responsible for protecting 
Legal Aid Ontario from fraud and billing errors 
by lawyers and other external service providers 
or legal aid clients. The department investigates 
alleged breaches of the Act, recovers overpayments 
made to lawyers and other service providers, and 
pursues recovery of amounts billed to certificates 
for which clients were not eligible. 

Investigations staff run computer analyses of 
lawyer billing and payments to identify inconsisten- 
cies, which then become the focus of investigations. 
To complete the investigation, staff must request 
from the lawyers documents verifying the work 
they completed. Such documentation can take a 
long time for the lawyers to produce, if it is ever 
produced at all. In 2010/11, investigations staff 
completed more than 250 solicitor and client file 
reviews, but were able to recover only $193,000. 
The amount recovered for 2009/10 was $110,000. 
According to Legal Aid Ontario, although manage- 
ment acknowledges that the recoveries may be low, 
just the existence of the department may well be 
acting as a deterrent against inappropriate billing. 

Beginning January 1, 2011, Legal Aid Ontario 
implemented a new policy requiring lawyers to 
submit these documents with their bills, with the 
exception of block-fee billing for criminal certifi- 



cates. Since the information will now be more 
accessible, staff expect this to improve the investi- 
gations process. For lawyers billing on a block-fee 
basis for criminal cases, staff will still need to ask 
them to submit documents supporting their work 
should this be deemed necessary. 

Investigations staff also need to obtain court 
information about specific case proceedings and 
outcomes in order to verify lawyer billings under 
investigation. Although Legal Aid Ontario has 
sought on-line access to the Ministry of the Attor- 
ney General's Integrated Court Offences Network 
(ICON) system for several years, that request 
has been denied. Instead, a Ministry liaison was 
appointed to handle their requests. Staff told us 
that there are often lengthy delays, and that some 
documents received are of poor quality so that the 
request must be made again. Our analysis showed 
that 20% of court information requests took longer 
than 30 days. We noted from our visits to both Que- 
bec and Manitoba that legal aid staff have on-line 
access to court information of this nature. 

Following our fieldwork, Legal Aid Ontario 
reached an agreement with the Ministry to receive 
monthly reporting on requested case details. How- 
ever, the agreement does not provide Legal Aid 
Ontario with on-line access to ICON. 



RECOMMENDATION 4 



To help ensure that internal controls over lawyer 
billing and payment processing are appropriate, 
Legal Aid Ontario should: 

• assess the recoveries achieved in the most 
recent year's billings using the new targeted, 
risk-based approach, and on that basis 
decide whether or not to proceed with an 
examination of billings from additional prior 
periods; and 

• assess the cost-effectiveness of its investiga- 
tion activities and continue to work with the 
Ministry of the Attorney General for timely 
access to court information that is needed 
for verifying lawyers' billings. 



Legal Aid Ontario 



| LEGAL AID ONTARIO RESPONSE 

Legal Aid Ontario agrees with the Auditor Gen- 
eral's recommendation. 

Legal Aid Ontario has developed a compre- 
hensive compliance plan that will continue to 
identify areas of payment risk for the organiza- 
tion and agrees to consider the probability of 
recoveries as part of its assessment. 

Legal Aid Ontario is improving the effect- 
iveness of its investigation group through the 
implementation of an automated case manage- 
ment program. Legal Aid Ontario has recently 
been given some access to court information by 
the Ministry of the Attorney General and con- 
tinues to work with the Ministry for improved 
access to court information needed to verify 
lawyers' billings. 

COMMUNITY LEGAL CLINICS 

Under the Act, community legal clinics are 
independent corporations governed and managed 
by boards of directors, and are accountable to Legal 
Aid Ontario. When deciding whether to provide 
funding to a clinic, the Act requires Legal Aid 
Ontario to consider the legal needs of individuals 
or communities the clinic serves, the clinic's cost- 
effectiveness and efficiency in providing legal aid 
services, the past performance of the clinic, and 
whether it is within Legal Aid Ontario's financial 
resources and priorities. In 2010/11, Legal Aid 
Ontario provided $65 million to 77 independent 
community legal clinics, which provided assistance 
to more than 155,000 low-income individuals. More 
than 85% of clinic expenditures are for salaries. 

We noted the following areas where oversight of 
the clinics could be improved: 

• Of the 77 clinics funded in 2010/11, all but 
two had submitted their budgets for approval 
on time as of February 1, 2010. However, 
we found that Legal Aid Ontario had not 
approved any of these budgets six months 



into the fiscal year (by September 30, 2010). 
By March 31, 2011, 18 budgets had still not 
been approved, although the clinics received 
their expected funding nonetheless. We are 
concerned that the value of the administra- 
tive effort to produce budgets is diminished 
when they are not analyzed and approved in a 
timely fashion. 

• The community legal clinics' financial eligibil- 
ity threshold for their clients receiving clinic 
services, which is different from Legal Aid 
Ontario thresholds for certificates, was last set 
in 1993 and has not been adjusted since then 
to account for general inflation. Clinics are not 
required to track the number of clients turned 
down, the reasons they were turned down, or 
whether they found alternative assistance — 
information that would be useful for identify- 
ing unmet needs. 

• Currently, clinics measure and report on 
outputs such as number of cases, number of 
public education sessions held, and number 
of referrals; however, there are no data on 
whether these outputs are achieving the 
desired program outcomes for clinic law mat- 
ters, such as successful appeals of disability 
income cases and landlord-tenant disputes. 
This had been noted as well in an earlier 
evaluation of clinics in 2004, by consultants 
hired by Legal Aid Ontario, and was noted 
again in 2008. In addition, Legal Aid Ontario 
was concerned about the accuracy of clinic 
productivity statistics, particularly with respect 
to whether client assists and the opening and 
closing of client case files were recorded in a 
consistent manner. We were advised that Legal 
Aid Ontario plans to address this issue through 
the development and implementation of a 
Clinic Information Management System, as 
noted in its business plan for 2011/12. 

• Legal Aid Ontario's internal auditors com- 
menced a clinic review program in 2009. So 
far, 42 clinics have been evaluated. During our 
audit, we were advised that the review was 
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put on hold pending a review of the program 
evaluation framework and that no further 
reviews were scheduled. 

Legal Aid Ontario issued a discussion paper 
to clinics in May 2010 outlining proposals for 
reducing overhead costs, such as rent and adminis- 
tration, in order to free up resources to serve more 
clients. Opportunities identified included regional 
co-ordination of services among clinics; shared 
space or co-location among clinics or with community 
agencies; shared services such as human resources, 
knowledge management, and finance; amal- 
gamation of clinics; and leveraging technology (for 
example, by providing Internet-based services to cli- 
ents) . At the end of our fieldwork, Legal Aid Ontario 
and the clinics were still assessing the options. 

For the most part, the clinic staff we spoke to 
expressed concern about Legal Aid Ontario's recent 
demands for greater efficiencies and about the level 
of support and communication the clinics receive 
from Legal Aid Ontario. Although the clinics are 
legally independent from Legal Aid Ontario, they 
are dependent on it for virtually all their funding 
and support, including information technology. For 
example, Legal Aid Ontario approves the clinics' 
client financial eligibility thresholds, budgets, salar- 
ies, rent, and reporting requirements. 

In essence, the clinics are accountable to Legal 
Aid Ontario, although on a day-to-day operational 
basis, they are accountable to their local boards 
of directors. This makes it challenging to propose 
and implement any system-wide changes because, 
although Legal Aid Ontario provides the funding, 
it is not always easy to obtain local buy-in for pro- 
posed changes. 



RECOMMENDATION 5 



To better address the legal needs of low-income 

individuals served by community legal clinics, 

Legal Aid Ontario should: 

• assess the impact of not increasing the clin- 
ics' income threshold for determining finan- 
cial eligibility since 1993; 



• consider requiring clinics to capture and 
report on the number of applicants who are 
denied assistance and the reasons they are 
denied; 

• improve the timeliness of the clinic budget 
review and approval process; and 

• develop and implement performance 
measures for clinics that are reflective of the 
outcomes achieved, together with a quality 
assurance program that includes the qual- 
ity of legal advice and services delivered to 
clinic clients. 

Legal Aid Ontario, in conjunction with rep- 
resentatives of community legal clinics, should 
assess the overall effectiveness of the local clinic 
structure and consider whether any changes 
are possible that would help serve more clients 
using available funding. 

| LEGAL AID ONTARIO RESPONSE 

Legal Aid Ontario agrees with the Auditor Gen- 
eral's recommendation and acknowledges that 
its approval process for clinic budgets needs to 
be faster. 

With respect to the issue of financial eligibil- 
ity for clinic law services, Legal Aid Ontario 
commits to assessing this matter in the way the 
Auditor General recommends. 

In July 2011, Legal Aid Ontario and the 
Association of Community Legal Clinics of 
Ontario (ACLCO) agreed upon an approach that 
is aimed at achieving $5.5 million in annualized 
administrative savings within the clinic system 
by 2015. Also, the ACLCO is leading a strategic 
planning initiative for the future of clinic law 
services. At the invitation of the ACLCO, Legal 
Aid Ontario will be participating in this process. 
In the course of these discussions, Legal Aid 
Ontario commits to raising the Auditor Gener- 
al's observations about the possibility of changes 
to the local clinic structure to serve more clients 
using available funding. 
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Additionally, Legal Aid Ontario is currently 
working with the clinics to develop and imple- 
ment a Clinic Information Management System. 
Performance measures are being developed as 
part of this project. This system will address the 
gaps identified by the Auditor General. 



INFORMATION TECHNOLOGY 

The Information Technology (IT) department of 
Legal Aid Ontario provides strategy, architecture, 
systems development, and project management 
for the organization, and supports approximately 
1,500 end users in more than 200 locations across 
the province, as well as about 4,700 panel lawyers. 
IT had 41 staff and five managers and an operating 
budget of $6.5 million in the 2010/11 fiscal year. 
Approximately $8 million is budgeted for upgrading 
infrastructure and application systems in the next 
two years. 

Ontario government standards require that 
departments assess threats and risks to which sensi- 
tive information, assets and employees are exposed; 
select risk avoidance options, implement cost-effect- 
ive safeguards, and develop comprehensive business 
continuity and disaster recovery plans. Information 
retained by Legal Aid Ontario on its clients is gener- 
ally considered solicitor-client privileged, requiring 
permission of clients before it is disclosed. We 
noted that, while there is a focus on information 
technology security and privacy management at 
Legal Aid Ontario through staffing, policies and 
procedures, and IT controls, there is no process to 
formally assess threats and risks associated with 
sensitive information, assets, and employees. We 
were advised by Legal Aid Ontario that the last 
privacy impact assessment was conducted in 2004, 
which would have been before they introduced the 
many new web-based systems in place and relocated 
their head office and most of their offices through- 
out the province. Without periodic assessments, 



management does not have objective assurance that 
sufficient safeguards exist to respond to privacy, 
security, and availability threats in the provision of 
information technology services. 

Performance measures are benchmarks for 
evaluating how information technology invest- 
ments can be more efficient and effective. The IT 
department does not currently report on perform- 
ance measures important to its operations and 
stakeholders. Such measures typically include 
system availability, response times to business 
requests, system changes to meet user needs, 
and costs of delivering services. IT has not yet 
developed such performance measures and targets 
in consultation with its internal and external stake- 
holders that reflect user needs. 



RECOMMENDATION 6 



To ensure that information technology systems 
meet privacy, security, and service level stan- 
dards, Legal Aid Ontario should: 

• periodically assess threats and risks associ- 
ated with its sensitive information and assets 
and take steps to manage the issues identi- 
fied; and 

• engage the users of the information tech- 
nology services in the development of key 
performance measures that would provide 
management with information on their 
progress in meeting user needs. 

| LEGAL AID ONTARIO RESP0NSI 

Legal Aid Ontario agrees with the Auditor Gen- 
eral's recommendation. 

Legal Aid Ontario performed a Threat Risk 
Assessment and Privacy Impact Assessment 
before it implemented its new web-based sys- 
tems in 2005. There is no evidence of privacy 
breaches associated with Legal Aid Ontario's 
information technology systems. 
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Ministry of the Attorney General 

Section! 

3.10 Office of the 



Children's Lawyer 



Background 



The Children's Lawyer is appointed by the Lieu- 
tenant Governor on the recommendation of the 
Attorney General. The Office of the Children's 
Lawyer (Office) is located in Toronto and has 
approximately 85 staff, including lawyers, social 
workers, and support staff. The Office also engages 
what it calls "panel agents" — approximately 440 
private lawyers and 180 clinical investigators across 
the province — on an hourly fee-for-service basis. 

The legal services the Office provides fall under 
various statutes and the Rules of Civil Procedure 
in the Superior Court of Justice. These services 
involve providing children under the age of 18 with 
legal representation for personal or property rights 
matters. Other parties whose interests might be 
at stake in a court proceeding involving children 
include the child's parents and relatives, Children's 
Aid Societies, and insurance companies. 

Personal rights proceedings include child 
protection cases and custody and access cases. The 
Office must provide legal representation for chil- 
dren in protection cases when ordered by the court 
under the Child and Family Services Act. For custody 
and access cases, the court may request under the 
Courts of Justice Act that the Office provide a child 
with legal representation, and the Office has discre- 



tion in accepting these cases. When it does accept 
them, it either provides lawyers to legally represent 
the child or has clinical investigators with expertise 
in social work help resolve the dispute and prepare 
reports for the court or involves both lawyers and 
clinical investigators, depending on the child's age 
and circumstances. 

The Office must represent children in property 
rights proceedings when appointed by the court or 
as required by legislation. In civil litigation cases, 
which consist mainly of personal injury actions, 
the Office may be ordered by a court to act as 
Litigation Guardian for the child where there is no 
parent, guardian, or other adult willing and able 
to pursue or defend a claim on behalf of the child 
and make decisions on his or her behalf. The Office 
also reviews proposed settlements referred by the 
courts in cases involving minors to assess whether 
they are in the best interests of the child and 
reports back to the court. In estate/ trust cases, the 
Office represents minor and unborn beneficiaries in 
matters such as challenges to the validity of a will, 
interpretation of a will, removal of executors and 
trustees, and other estate administration matters. 

The Office is part of the Ministry of the Attorney 
General (Ministry). For the 2010/11 fiscal year, the 
Office's expenditures were approximately $32 mil- 
lion, including $22 million for external professional 
services and $9 million for internal staffing costs. 
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The Office is unique in Canada for the broad 
range of legal and other services that it provides 
to children. Although there are other agencies in 
Ontario that offer support services to children, they 
generally do not have the mandate or funding to 
provide children with their own independent legal 
representation for court proceedings. 

The Office accepts about 8,000 new cases per 
year and as of March 31, 2011 had more than 
11,000 open cases. Figure 1 illustrates the types of 
open cases as of March 31, 2011. 



Audit Objective and Scop 



Our audit objective was to assess whether the Office 
of the Children's Lawyer had adequate policies, 
procedures, and systems in place to: 

• serve the personal and property interests of 
children in accordance with legislative and 
court requirements; and 

• measure and report on its efficiency and 
effectiveness in doing so. 

Senior management reviewed and agreed to our 
audit objective and associated audit criteria. 

The scope of our audit included interviews with 
ministry officials and an examination of files, docu- 
mentation, and policies in use at the Office's only 
location, in Toronto. We contacted stakeholders 
from the private bar and other agencies that provide 
children's services to discuss their perspectives on 
the services provided by the Office. We also engaged 
as advisors independent experts in legal services 
and child and youth services and researched how 
other jurisdictions provide legal representation to 
children. In addition, we obtained input from the 
Chief Justice of the Superior Court of Justice and 
the Chief Justice of the Ontario Court of Justice. 

We did not rely on the Ministry's internal 
auditors to reduce the extent of our audit work, 
because they had not conducted any recent audits 
of the Office. However, we were able to reduce our 
work on financial controls, particularly with respect 



Figure 1: Open Cases at the Office of the Children's 
Lawyer as of March 31, 2011 

Source of data: Office of the Children's Lawyer 
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to payments to service providers, because we 
examine these annually as part of our annual audit 
of the Office of the Children's Lawyer's financial 
statements. 




Ontario legislation and the province's courts 
provide children in need of protection of their per- 
sonal and property rights with independent legal 
representation through the Office of the Children's 
Lawyer. Demand for the Office's legal and clinical 
investigation services is significant. As well, the 
Office is unique in that no other jurisdiction in 
Canada provides children with the same range of 
centralized legal services. Overall, the legal and 
investigative work done by the Office is valued 
by the courts, children, and other stakeholders. 
However, these services are often not assigned or 
delivered in a timely enough manner. 

We also found that the Office's case manage- 
ment system was not meeting its information 
needs and that it did not have an adequate process 
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in place for evaluating the cost-effectiveness of 
its operations. For example, the Office had not 
adequately analyzed why its payments to panel 
agents had increased by more than $8 million, or 
60%, over the last 10 years even though new cases 
accepted decreased by 20% and the Office's overall 
active caseload did not change significantly over the 
same period. 

We identified several other areas where the 
Office's systems, policies, and procedures war- 
ranted improvement, as follows: 

• In the 2010/11 fiscal year, the Office exercised 
its discretion to refuse more than 40% of 
child custody and access cases referred to 

it by a court. We found that the Office had 
not adequately assessed the impact of these 
refusals on the children and courts. The 
Office's decisions were based on reasons to 
refuse a case rather than reasons to accept a 
case based on the best interests of the child. 
As well, many of the decisions to refuse cases 
were made primarily because of limited finan- 
cial resources. In addition, it had not explored 
the reasons for fairly significant regional fluc- 
tuations of between 29% and 50% in refusal 
rates across the province. 

• Although the Office has substantially reduced 
the time it takes to accept or refuse custody 
and access cases, from 68 days in 2008/09 to 
39 days in 2010/11, it still is not meeting its 
21 -day turnaround target. Also, once a case 
was accepted, it took more than eight weeks 
to assign almost 50% of cases to staff or an 
agent before work could commence. Improved 
information systems would help ensure that 
the causes of these delays are better identified 
for corrective action. 

• In a custody and access case where the Office 
is providing the court with a Children's 
Lawyer Report detailing its investigation and 
making recommendations to the court on the 
custody of and access to a child, the Family 
Law Rules require that it do so within 90 
days. However, the Office met this deadline 



less than 20% of the time and did not have 
any formal strategy in place to improve its 
performance in this area. 

• The Office had a sound process for ensuring 
that personal rights lawyers and clinical inves- 
tigators were well qualified and selected fairly. 
However, there was no open selection process 
in place for the almost 100 property rights 
lawyers the Office had under engagement at 
the time of our audit. 

• The Office permits property rights panel 
lawyers to charge a rate of up to $350 an hour 
when recovering their costs from a child's 
estate or trust or settlement funds. Yet if the 
same lawyers charge their services directly to 
the Office, they are paid $97 an hour. 

• The Office's programs for reviewing the qual- 
ity of the work performed by panel agents did 
not include an assessment of whether the fees 
charged were reasonable. 

• A new case management system, scheduled 
for November 2011 and estimated to cost 
$3.8 million, might not meet all of the 
Office's key information needs and functional 
requirements. 

• There were no formal protocols for transition 
planning and support to assist children (other 
than those who are mentally incapable) with 
the management of their ongoing civil law- 
suits or estate matters when they turn 18 and 
no longer qualify for the Office's services. 

• The Office did not have objective measures 
to assess and report on its performance, nor 
were there formal, regular processes for 
assessing whether stakeholders, including 
children, were satisfied with the services 
provided. 

We did note that the Office had established 
quality assurance processes and training programs 
to help ensure that legal and clinical investigative 
services were being consistently and competently 
delivered. 
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| OVERALL OFFICE RESPONSE 

The Office of the Children's Lawyer is com- 
mitted to continuing to provide the children it 
serves with the highest-quality legal and clinical 
services in a wide range of court matters within 
its mandate. 

The Office has begun a multi-year organ- 
izational transformation to ensure optimal 
alignment of its resources with the needs of 
the children it serves. A key part of this change 
initiative is the modernization of case manage- 
ment technology and business processes that is 
being implemented in phases and will enable 
increased efficiency and responsiveness in deliv- 
ering high-quality and timely services. 

Improvements are being made in areas 
including: 

• information and case management; 

• monitoring and tracking; 

• empanelment of, and payments to, its legal 
agents; 

• financial forecasting; 

• staff and panel agent training and 
development; 

• stakeholder outreach; and 

• key performance indicators. 

The Office welcomes and supports the find- 
ings and recommendations as it continues to 
move forward to increase its effectiveness in 
delivering its services to the children of Ontario. 



Detailed Observations 



INTAKE AND REFERRAL OF CASES 

Decision to Accept or Refuse a Custody and 
Access Case 

The Courts of Justice Act gives the Office discretion 
to decide whether it will provide representation 
to children in custody and access cases when it 



is asked to by the court, and it has established an 
intake process for reviewing and deciding whether 
to accept these types of cases. In the 2010/11 fiscal 
year, it received almost 4,480 custody and access 
cases and rejected about 1,820 of them, or 41%. 
Figure 2 shows that the percentage of custody 
and access files refused over the last five years has 
ranged from 41% to 55%. 

More than 10 years ago, the Office developed 13 
criteria that its intake staff and senior management 
use to decide whether to refuse to provide services 
in referred custody and access cases. To assess a 
case against its refusal criteria, the Office requires 
a copy of the court order requesting its services, 
which is usually forwarded to it by the court or an 
involved party, such as a parent's counsel. It also 
requires the submission of standardized intake 
form that provides information about the family's 
history, the situation, and the relationship between 
the parents and children involved. Parties must 
submit their forms to the Office within 10 calendar 
days of the date of the court order or the case may 
be refused. The Office's intake clerks use the forms 
and collateral information, such as Children's 
Aid Society investigations or medical reports, to 
prepare a summary of the case and make a recom- 
mendation to accept or refuse the case to either 
the Personal Rights Legal Director or the Manager 

Figure 2: Number and Percentage of Custody and 
Access Cases Accepted and Refused, 2006/07- 
2010/11 

Source of data: Office of trie Children's Lawyer 
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of Clinical Services, who make the final decision. 
Generally, for accepted cases, those that involve 
children under the age of 12 are recommended 
for clinical investigator services, and those that 
involve children who are aged 12 and over, who 
are more likely and able to express their views and 
preferences to lawyers, are recommended for legal 
representation. 

We were advised that the 13 refusal criteria are 
used to ensure that the Office provides its services 
only in cases where it believes that it could add 
value. Nonetheless, we questioned whether the 
Office ought to establish criteria that set out when a 
case should be accepted and that focus on ensuring 
that it accepts a case when it is in the best interest 
of the child to do so. Acceptance criteria would 
enable the Office to better track the common rea- 
sons for being involved in a custody and access case 
as well as the benefits provided to children. The 
Office has not done any studies or assessments, and 
had no other mechanisms in place, to determine the 
impact on children and the courts of its refusing, 
over the last five years, an average of 44% of the 
custody and access cases referred by the courts. 

We were advised that the Office's decisions to 
refuse custody and access cases were all made from 
its head office in Toronto and not on a regional 
basis, nor are its budgets based on regions. How- 
ever, we noted that the Office had not determined 
the reasons for inconsistent refusal rates for its nine 
regions throughout the province. Figure 3 indicates 
that the average refusal rates varied from 29% in an 
eastern region to 50% in its northern region. 

Although 12 of the 13 refusal criteria are pub- 
lished on the Ministry's website, the Office is not 
required to disclose its reason for refusal to the 
parties in a particular case. We were advised that 
this is a common complaint of parties whose cases 
have been refused. However, the Office will inform 
the parties if a case is refused for the following 
reasons: an intake form was not received; one of 
the parties resides outside of Ontario; there is a 
restraining order prohibiting contact between one 
of the parties and the child (ren); supervised access 



Figure 3: Average Refusal Rate for Custody and Access 
Cases, by Region (%) 

Source of data: Office of the Children's Lawyer 
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has been ordered but has not yet commenced; 
Children's Aid Society (CAS) has not responded to a 
request for information regarding its involvement; 
or the child (ren) are in the care of CAS. Parties may 
request a reconsideration of their case after it has 
been refused by the Office. 

Based on the refusal criteria, the Office's inter- 
nal reports indicate that more than 90% of the 
custody and access cases that it refused were turned 
down for the following reasons: 

• 38% are refused because there is insufficient 
information to evaluate the case; 

• 36% are refused because other resolution 
efforts should have first occurred but have not 
been attempted; and 

• 17% are refused because the Office deems 
that the child's situation would not be 
improved by the Office's involvement. 

The Office records in Case Track, its computer- 
ized case management system, the reasons for 
refusing a case according to one of the 13 intake 
criteria. We noted some inconsistencies between 
the reasons for refusal that were documented in 
the file compared to those documented in the 
database. In addition, in some instances where it 
had been recorded in the database that a case had 
been refused, the file noted that the Office had 
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actually accepted the case and provided services. 
The documentation of how a decision was arrived 
at to refuse a case could also be improved. For 
example, more than a third of cases were refused 
because other resolution efforts (such as media- 
tion, clinical assessments, or family counselling) 
should have occurred and had not been attempted. 
However, almost half of the cases we reviewed 
that were refused for this reason did not indicate 
the specific resolution efforts that should have 
been considered, nor was it evident from the file 
summaries. Without clear documentation of how 
decisions are made, there is a risk that intake staff 
may not be adequately and consistently assessing 
children's needs, and this would not be apparent 
from any supervisory or management review of the 
case documentation. 

Senior management at the Office informed us 
that one common reason for refusing a custody and 
access case was a lack of available funding for the 
Office to accept more cases; however, this reason 
was not tracked, nor is it one of the 13 refusal 
criteria. For cases that would otherwise be accepted 
(based on the refusal criteria) had funds been avail- 
able, we were told that another reason for refusal 
is selected from one of the 13 refusal criteria. The 
Office uses a forecast model to ensure that expendi- 
tures for the year stay within budget. Because 
custody and access cases are the most significant 
type of cases the Office has the discretion to refuse, 
budget considerations affect how many cases at any 
given time can be accepted. However, the number 
of cases rejected due to funding limitations is not 
specifically monitored or even known. This would 
be useful information to communicate to senior 
ministry decision-makers as part of the Ministry's 
annual budgeting process. 

Timeliness of Decisions 

The Office has identified the issue of delays in the 
intake process as a concern to its stakeholders. It 
completed internal reviews in 2007 and 2008 to 
seek ways to improve the process. The 2007 report 



made 15 recommendations, including changes to 
the intake form, training for staff, granting more 
authority to intake staff to refuse files, replacing the 
Case Track system, and reassessing the 21-day target 
turnaround time for deciding whether to accept or 
refuse a case. At the time of our audit, the Office 
had implemented or partially implemented seven 
of the 15 recommendations and was working to 
address those that remained. For instance, intake 
staff report weekly to management the number of 
cases refused and accepted, the number still waiting 
to be processed, and the reasons for refusing cases to 
help identify outstanding cases and causes of delay. 

We noted that the Office has made headway in 
reducing delays in decision times, but not enough 
to achieve the 21-day target turnaround time. Our 
sample showed that average turnaround time sig- 
nificantly improved between 2008/09 and 2010/11, 
falling from 68 days to 39 days, but it was still about 
85% higher than the target. The Office records the 
date a file is received and the date a recommendation 
to accept or refuse a case is made by an intake clerk, 
but it does not record the date the Personal Rights 
Legal Director or Manager of Clinical Services makes 
the final decision. It is difficult to accurately assess 
where bottlenecks are occurring if this information is 
not tracked. Using data from Case Track that includes 
the date a case is received and the date that it is for- 
warded to the Director or Manager for a decision, 
our analysis indicated that over the last three years: 

• 17% of cases took 21 days or less to be 
forwarded; 

• 67% took between 22 days and 56 days to be 
forwarded; and 

• 10% took 57 or more days, or more than eight 
weeks, to be forwarded. 

We could not determine time frames for the 
remaining 6% of cases because of missing or incon- 
sistent information in Case Track. 

Case Assignment 

The Office's Accounts and Referrals unit is respon- 
sible for assigning protection cases when a court 
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order is received requiring the Office's involvement 
and for assigning accepted custody and access 
cases. Cases outside of the Toronto region are 
generally assigned to panel agents residing in the 
local community Cases in the Toronto region are 
assigned to both in-house staff and panel agents. 

Panel agents have voiced concerns about unfair 
distribution of cases across a particular region. 
We also noted wide disparities in the number of 
cases being assigned to agents within a geographic 
region. Although an agent may have a good reason 
for not carrying more cases (such as being newly 
empanelled or having other workload), the Office 
has an inadequate system in place to monitor and 
track panel agents' workload or their reasons for 
rejecting cases. Our analysis indicated that in one 
region there were 22 active legal agents, each 
carrying an average caseload of 17 files; six of these 
agents were carrying fewer than five files, and four 
of them were carrying 30 or more. The Office could 
more appropriately assign cases if it tracked and 
took into consideration the current caseload of each 
of its panel agents. 

Furthermore, the Office has a policy requiring 
prior authorization for legal agents to be assigned 
more than 50 files at a time and for clinical agents 
to be assigned to prepare more than two Children's 
Lawyer Reports per month. (If the parties do 
not resolve the dispute, the clinical investigator 
prepares a Children's Lawyer Report under the 
Courts of Justice Act for the court that assesses the 
children's wants and needs and the family's circum- 
stances.) Accounts and Referrals staff informed us 
that there is no documentation in the personnel 
files or any notation in Case Track to indicate a 
legal agent having received authorization to carry 
more than 50 cases at a time. Accounts and Refer- 
rals staff told us that they rely on the supervisors 
in other areas of the Office who monitor agents to 
advise them that an agent should not be assigned 
anymore files. As of April 2011, there were 15 
legal agents who were carrying more than 50 files. 
One agent had a caseload of 123 files. However, 
no documentation was on file indicating that the 



required prior authorization had been given for 
these agents to carry more than 50 cases. 

Similarly, there was no documentation or 
notation in Case Track to indicate a clinical agent 
receiving authorization to be assigned more than 
two Children's Lawyer Reports per month. We were 
informed that, although supervisors monitor case- 
loads through regular file reviews of each agent, 
the Accounts and Referrals staff who actually assign 
the cases did not actively keep track of agents' case- 
loads. We noted that as of April 2011, there were 
eight clinical agents with a caseload of 10 or more 
Children's Lawyer Reports. 

We also found that there was no tracking of files 
waiting to be assigned at Accounts and Referrals. 
Clerks report monthly on the number of unassigned 
files, but they do not report how long those files 
have been left unassigned. We were informed that 
a case may take up to three weeks to be assigned 
because there are no agents willing or able to 
take it. Possible reasons for this include conflict of 
interest and excessive workload. The Office does 
not formally keep track of panel agents' refusal of 
cases and their reasons for refusing a case. It has 
attempted to address delays by recently imple- 
menting a new procedure that requires clerks to 
bring a file to the attention of the manager if it has 
been unassigned for two weeks. In the meantime, 
the Office advised us that child protection cases 
are a high priority for the Office and are assigned 
to an agent within five business days of acceptance 
in 80% to 90% of cases. However, we found long 
delays for custody and access cases to be assigned, 
as follows: 

• 7% of cases took 28 days or less to be 
assigned; 

• 36% took between 29 and 56 days to be 
assigned; and 

• 47% took 57 or more days, or more than eight 
weeks, to be assigned. 

We could not determine time frames for the 
remaining 10% of cases because of missing or 
inconsistent information in Case Track. 
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RECOMMENDATION 1 



To ensure that its intake and referral services 
make appropriate and timely decisions on 
whether to accept or reject a custody and access 
case and whom to assign a personal rights case to, 
the Office of the Children's Lawyer (Office) should: 

• establish criteria for accepting cases based 
on the best interests of the children involved 
and the benefits provided by the Office's 
involvement, and track these reasons for 
accepting them — the reasons for refusing 
cases should also continue to be tracked, but 
recorded more accurately, including noting 
when funding limitations affect the decision 
to refuse a case; 

• examine the impact on children and the 
courts of its refusal rate of more than 40% 
for custody and access cases referred to the 
Office by the courts; 

• monitor the number of cases assigned to 
each in-house lawyer and panel agent, and 
ensure that higher-than-normal caseloads 
receive the required authorizations; and 
establish recording and reporting systems 
that allow management to adequately track 
and monitor the time it takes to accept or 
reject a custody and access case as well as to 
assign an accepted case, and use this infor- 
mation to identify any systemic reasons for 
delays. 



OFFICE RESPONSE 



The Office strives to be responsive to the needs 
of children, parents, other parties, and the 
family courts and to inform them in a timely 
way about whether custody and access cases 
have been accepted. 

The Office is also committed to providing 
staff with the tools they need to make appro- 
priate case acceptance/refusal decisions and 
to accurately record the reasons for decisions 
taken. 



The Office is taking steps to: 

• articulate and record in more detail the 
criteria used by the Office when accepting or 
refusing a custody and access case, including 
specifying when funding limitations are a 
factor; 

• communicate to senior management in the 
Ministry the number of custody and access 
cases accepted and refused; 

• reduce the current turnaround times for 
communicating a decision to accept/refuse a 
case; 

• monitor and authorize, when appropriate, 
panel agent caseloads that are beyond estab- 
lished thresholds; and 

• analyze and measure case flow to identify 
systemic issues affecting the management 
of cases from the time a case is opened to its 
assignment, if accepted. 



TIMELINESS OF COURT REPORTS 

The Family Law Rules of the Superior Court of Jus- 
tice require that the Office file a Children's Lawyer 
Report with the court within 90 days of serving 
notice to the parties that an investigation is to be 
conducted. We were informed that the Office does 
not view the 90-day time frame to be realistically 
attainable and that it has attempted in the past to 
extend this time frame through discussions with 
the Family Rules Committee but was unsuccessful. 
The Office monitors the number of reports that 
meet the 90-day requirement and the number of 
reports that were completed within 120 days. Since 
April 2006, the Office has reported that less than 
20% of assigned reports were filed within 90 days, 
with an additional 22% of reports filed within 120 
days. We also noted that 25% of reports took more 
than 180 days to complete, with the longest taking 
almost 400 days. The Office had not established an 
action plan to improve its performance in meeting 
the 90-day deadline. 
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RECOMMENDATION 2 



To help improve its performance in meeting a 
regulated 90-day deadline for filing Children's 
Lawyer Reports with the court, the Office of 
the Children's Lawyer should establish a formal 
strategy that addresses the changes needed to 
its systems and procedures in this area. 



OFFICE RESPONSE 



The Office remains committed to delivering its 
Children's Lawyer Reports to the parties and 
the courts in a timely manner. The preparation 
of these reports is time-intensive and requires 
meetings with the parties, meetings with 
the child (ren), observing the parties and the 
child (ren), obtaining information from several 
external sources, and drafting the report. 
Accordingly, it is often difficult to meet the 
90-day timeline. 

To improve timeliness, the Office is examin- 
ing and analyzing the obstacles to meeting the 
90-day timeline. It is also exploring alternative 
forms of fact-gathering and report-writing, both 
within the Office and with stakeholder partners. 
An action plan is being developed, geared spe- 
cifically to reducing impediments to meeting the 
90-day timeline. 



PANEL AGENTS 

Empanelment Process 

The Office uses "panel agents" — lawyers and 
clinicians (that is, social workers or psychologists) 
working in private practice — to supplement its 
own staff and to provide services throughout the 
province. An empanelment process is used to select 
and prequalify lawyers and clinicians, who are then 
enrolled to a list, also called a panel, and who can 
then be assigned cases in their region. 

We found that a comprehensive empanelment- 
selection process was in place for lawyers and 



clinicians hired for personal rights cases, but there 
was no equivalent process for the lawyers the Office 
used in property rights cases. 

The Office advertises its empanelment process, 
and interested lawyers and clinicians submit 
applications and references. The Office requires 
that personal rights legal and clinical agents have 
sufficient credentials, knowledge, experience, and 
interpersonal skills to deal effectively with children 
and families. Both legal and clinical applicants 
must sign an agreement listing the undertakings 
expected of the agent if he or she is selected for 
the panel — agreeing to comply with Office policies 
and procedures, lawyers being a member in good 
standing with the Law Society of Upper Canada, 
submitting invoices on time, attending training, 
and accepting all cases assigned except where there 
is a conflict of interest. 

The Office's panel agents for personal rights 
cases are retained for a three-year term. They may 
leave or be removed from the panel at any time, 
and new agents may be hired in-term, if required. 
At the end of the empanelment period, agents must 
reapply if they wish to remain on the panel. At the 
time of our audit, there were about 335 active panel 
lawyers carrying more than 7,200 cases, and about 
180 active clinical agents carrying about 1,150 
cases. We concluded that this was a sound process. 

However, the Office has not established a similar 
process or criteria for its property rights agents. 
At the time of our audit, there were 98 private 
lawyers retained by the Office for property rights 
cases, an increase of 17 lawyers or 17% from the 
previous year. The Office informed us that it seeks 
out lawyers in private practice who have skills and 
experience in conducting estate and civil litigation 
cases. The Office also relies on lawyers that have 
established good working relationships based on 
previous services provided. Nevertheless, a more 
formal and open empanelment process for property 
rights lawyers would be more consistent with the 
general principles of a transparent and fair procure- 
ment process. 
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Tariff Rates 

The Office sets maximum amounts for the rates and 
hours that personal rights panel agents are allowed 
to charge for their services. Any service hours 
charged over the maximum require prior authoriza- 
tion by the Office. In general, lawyers may charge 
up to 30 hours for the first year of a case, and up to 
15 hours for the second and each subsequent year, 
which is increased to 40 and 20 hours, respectively, 
if four or more children are involved. They are also 
allowed additional hours if the case proceeds to 
trial. Clinical panel members can charge up to 30 
hours for preparing a Children's Lawyer Report or 
in the first year in a case requiring clinical assist- 
ance, and up to 15 hours in each subsequent year. 

Historically, changes to the tariff rates paid 
to Legal Aid Ontario lawyers have been followed 
within a few months by a matching increase in the 
Office's legal tariff rates. 

On January 25, 2010, the Attorney General 
announced that the province was going to increase 
the rates for Legal Aid Ontario lawyers. One of the 
changes introduced under the agreement was an 
increase in hourly fees for criminal, family, immi- 
gration/refugee, and mental-health lawyers by an 
average of 5% per year for the next seven years. 
As of April 1, 2011, lawyers working for Legal Aid 
Ontario receive an hourly rate of around $112, 
compared to a rate of $97 for those hired by the 
Office. However, the Office's last tariff increase was 
approved more than three years ago by the Ministry. 

Stakeholders advised us that they found the dif- 
ference in rates for similar services unfair because 
in many cases in the same courtroom parents may 
be represented by Legal Aid Ontario lawyers while 
their children are represented by the Office's panel 
lawyers at a lower rate. 

The Children's Lawyer has made a request to 
the Ministry to match the rates paid by Legal Aid 
Ontario. The Office estimated that the financial 
impact of this proposed tariff increase would be a 
10% rate increase effective November 1, 2010, and 
would result in the Office requiring approximately 



$732,000 in additional funds for the 2010/11 fis- 
cal year and 5% per year over the subsequent five 
years. At the time of our audit, the Office's request 
had not been approved. 

Property Rights Legal Fees 

When the Office represents a child in an estate 
matter, the services will be paid for by the Office 
at the tariff rate unless the fees can be paid by 
another party to the litigation or out of the estate/ 
trust, or the settlement. When the fees are to be 
paid by another party, or from the estate/trust or 
settlement, the property rights staff lawyers review 
the accounts and the court approves the payments 
from the other party or the child's funds. In civil 
cases, when damages are paid to a child, such as for 
accident claims, panel agents are instructed to seek 
recovery of their costs from another party to the 
litigation whenever possible. If it is not possible to 
recover costs from another party, fees are paid from 
the settlement, after being approved by the court. 

We noted that the Office was paying lawyers 
it engaged for property rights cases $97 per hour 
when the Office paid for the services, but had 
established a policy that allowed lawyers to "charge 
their usual hourly rate" up to a maximum of $350 
per hour ($300 per hour before June 2010) when 
a child paid for the services from the estate/trust 
or settlement. The Office advised us that a higher 
rate was established as a means of attracting and 
retaining property rights counsel to do work for it. 
In 2002, the rate was capped at $300 per hour. In 
our discussions with them, Office staff noted that 
they hire expert lawyers to handle more complex 
cases, sometimes on a contingency-fee basis, and 
this expertise necessitates higher fees. In our view, 
this still does not explain why the Office permits 
lawyers to charge only the tariff rate when the 
Office is paying and to charge a rate that is more 
than three times higher when they are being paid 
from the child's funds. 

The Office informed us that it reviews all cases 
that have settled with payments to agents to ensure 
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that the payments made on behalf of the child are 
acceptable and that these fees are approved by a 
court. We requested information on how much has 
been paid to lawyers from estates/trusts and settle- 
ments at the rate above the tariff, but the Office 
does not keep track of this information because the 
payments do not come from its budget. Therefore, 
the Office could report only the amount that it 
paid out to property rights agents at the tariff rate, 
which was $354,000 in 2010/11. 

Because the Office pays the lower tariff rate to 
many different panel lawyers for property cases, it 
should have a good understanding of which of these 
lawyers are most capable of handling the more com- 
plex cases. We suspect that many of these lawyers 
might well, if offered the opportunity, be willing 
to undertake property cases on behalf of a child's 
estate for significantly less than $350 an hour. 

Payments 

Legal and clinical agents for personal rights cases 
are required to submit invoices to the Office at 
least three times per year but not more than once 
a month and not for less than $100 for services 
rendered. 

Accounts and Referrals clerks receive all per- 
sonal rights agents' invoices and manually enter 
the payments in Case Track. The clerks check 
the invoices for accuracy, correct tariff rates, and 
approved disbursements and ensure that any 
amounts over the tariff rate include documenta- 
tion of prior authorization before approving them 
for payment. However, the clerks do not have the 
knowledge of the cases to be able to assess whether 
the amounts billed are reasonable given the servi- 
ces provided. A supervisor, staff lawyer, or clinical 
investigator reviews panel agents' files as part of the 
Office's quality assurance program, but this review 
does not include an assessment of invoices either 
before or after payment to ensure that the charges 
were reasonable. Office staff informed us that 
supervisors have occasionally conducted ad hoc file 
examinations where billings that were considered 



higher than the average amounts were reviewed 
for reasonableness of services provided, but these 
examinations have not been consistently done and 
there was no record of specifically which files were 
reviewed or any documentation of the procedures 
followed when they are done. 

Until 2010, invoices pertaining to a particular 
case were not centrally stored; rather, the invoices 
relating to the case could be located in several 
different batches of payments, making it labour- 
intensive to locate all of an agent's invoices for a 
single case to review the billings after payments 
had been made. Although invoices are now filed 
by case, processes for regularly examining pay- 
ments have still not been established. At the end of 
our audit fieldwork, the Office told us that it had 
been informed by an outside source about possible 
fraudulent billings by a panel agent that may have 
taken place over the past 10 years and, although it 
was still too soon to know the extent of the billing 
irregularities in that particular case, it had initiated 
an investigation of its payment practices. We also 
noted from our discussions with Legal Aid Ontario 
that it was implementing a process for conducting 
regular post-payment examinations of its panel law- 
yers' invoices to ensure that the payments made to 
the panel lawyers were appropriate and reasonable 
in relation to the work done. 

The Office informed us that agents regularly 
complain about the length of time it takes for their 
invoices to be paid. One of the Office's performance 
measures is to have 80% of invoices paid within 30 
days. The Office has reported its difficulties in meet- 
ing this target — the percentage of invoices paid 
within 30 days fell from 78% in 2006/07 to just 
26% in 2009/10. The Office informed us that it is 
working to address delayed payments and has hired 
contract staff for 2010/11 and 2011/12 to clear 
the backlog of invoices waiting to be processed. 
This resulted in 71% of invoices being paid within 
30 days for the 2010/11 fiscal year. However, the 
Office has not determined whether it is possible to 
change its current business processes to expedite 
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invoice processing without having to resort to the 
periodic use of contract staff. 

Block Payments 

Block fees are fixed fees that are paid for common 
types of services. We discovered that, to reduce 
administrative costs and provide more financial 
certainty, Legal Aid Ontario was changing to a 
block-fee framework for many legal services rather 
than paying by the actual number of hours incurred 
on a case. To implement block-fee payments, Legal 
Aid Ontario reached an agreement with its legal 
stakeholders to pilot a new payment method. The 
first and second phases of its block-fees program 
were implemented in May 2010 and May 2011. 

The Office informed us that it was not formally 
considering other billing structures, such as block 
fees or alternative payment arrangements, for 
personal rights cases. We were also advised that the 
Office had implemented a block-fee arrangement 
with a firm to handle aspects of its 2003-2009 
property rights cases; however, the Office did 
not have any information or analysis on the cost- 
effectiveness of this arrangement. 

The Office's annual review of tariff fees paid to 
panel agents includes a review of the total hours 
paid for and total disbursement amounts above 
the standard tariff hours allowed. However, we 
noted that the Office's billing system is not capable 
of reporting on the number of hours and amounts 
billed compared to the allowable maximums and 
other similar analyses. Access to information on the 
extent to which particular types of cases require 
more or less than the standard tariff hours, or on 
whether certain agents consistently require more or 
less time than the tariff allows, would be useful in 
evaluating allowable tariff hours and different pay- 
ment frameworks. 



RECOMMENDATION 3 



To ensure that it has adequate systems, policies, 
and procedures for acquiring, reimbursing, and 



managing its legal and clinical panel agents, the 
Office of the Children's Lawyer (Office) should: 

• develop a more open empanelment process 
for lawyers hired for property rights cases 
similar to the sound process already in place 
for personal rights panel agents; 

• further consult with the Ministry of the 
Attorney General on establishing a process 
whereby the tariff rates for panel lawyers 
would be the same as the rates paid by Legal 
Aid Ontario; 

• assess whether alternatives may be available 
to retain appropriate lawyers for property 
rights work to enable at least some reduction 
in the current significant premium rates 
being paid for services billed directly to the 
estates/ trusts or out of settlement funds 
belonging to the child; 

• implement better systems and procedures for 
scrutinizing legal fees, such as post-payment 
examinations and assessing the reasonable- 
ness of invoices, and for paying them within 
targeted time periods; and 

• in conjunction with its stakeholders, 
research and evaluate alternative methods 
of payment to its panel agents, such as block- 
fee payments, that would increase financial 
certainty in payments and reduce adminis- 
trative processing requirements and costs for 
the Office. 

| OFFICE RESP0NSI 

The Office values the experience, knowledge, 
and commitment of its legal and clinical panel 
agents and is committed to providing high- 
quality services for children in a cost-effective 
manner. Accordingly: 

• The Office will develop a fair and open pro- 
cess for the empanelment of qualified agents 
to provide representation for the Children's 
Lawyer in property rights cases across the 
province, similar to the process already in 
place for personal rights panel agents. 
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A Ministry-approved increase in the tariff 
rates for panel lawyers to match the rates 
paid by Legal Aid Ontario, retroactive to 
July 1, 2011, has been implemented. The 
Office will consult with the Ministry on 
establishing a process for the timely con- 
sideration of requests for future tariff -rate 
increases and possible synchronization with 
future Legal Aid Ontario rate increases. 
The Office will canvass other Ontario Public 
Service and broader-public-sector organiza- 
tions and consult with stakeholders to assess 
whether there are suitable alternatives to 
the current retainer model that can be used 
in the small specialized portion of property 
rights cases where panel agents are retained. 
Such an approach must, however, maintain 
the high-quality legal representation that the 
Office currently provides to children. 
The Office's new case management system, 
Children Information and Legal Database 
(CHILD), scheduled for phase-one implemen- 
tation in December 2011, will automate and 
improve the Office's information technology 
systems, as well as its processes for acquiring, 
reimbursing, and managing its legal and clin- 
ical panel agents. The second phase of imple- 
mentation of CHILD, scheduled for spring 
2012, will allow for electronic billing and 
more timely and efficient invoice payments. 
The Office will improve current auditing and 
assessment of agent bills-of-account for rea- 
sonableness and compliance with legal and 
clinical tariffs, as well as consider alterna- 
tive methods of review and payment of its 
fee-for-service panel-agent invoices, such as 
post-payment examinations. It is anticipated 
that the portal component of the new system 
will significantly automate the submission 
and processing of agent payments, and 
reduce the Office's administrative costs. 
The Office will also examine alternative bill- 
ing methods, such as block-fee payments, as 
part of the new systems evaluation. 



PROGRAM COSTS 

Cost Analysis and Forecasting 

We found, as shown in Figure 4, that the Office's 
expenditures had increased at a substantially 
higher rate than its underlying service volumes over 
the past 10 years and that the Office needed better 
management information to allow it to ensure and 
demonstrate the cost-effectiveness of its operations. 
For instance, over the 10-year period from the 
2001/02 to the 2010/11 fiscal years, we noted the 
following: 

• Overall program expenditures increased from 
$20.6 million to almost $32 million, or by 55%. 

• Payments made to panel agents, which 
account for approximately 70% of the Office's 
total budget, increased from about $13.6 mil- 
lion to $21.7 million, or by 60%. 

• New cases accepted decreased by about 20%, 
and the Office's overall active caseload did not 
change significantly. 

Although some of these changes can be 
explained by three tariff-rate increases over this 
period, the Office had not conducted any formal 
analysis to identify the extent to which other fac- 
tors — such as more complex and time-consuming 
cases or process inefficiencies — contributed to the 
disproportionate cost increases. 

Figure 4: Accepted New Cases and Program Costs, 
2001/02-2010/11 

Source of data: Office of the Children's Lawyer and Public Accounts of Ontario 
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We also found that the Office was not tracking 
and monitoring its case costs as fully as it could. For 
instance: 

• The Office does not know the cost of handling 
a personal rights case from beginning to end. 
Instead, to determine its budget the Office 
calculates the average cost of all the ongoing 
personal rights cases in a particular year by 
totalling the amounts paid out that year div- 
ided by the total number of ongoing cases at 
the end of the year. The Office also uses aver- 
ages to estimate case cost per agent in a given 
year, and has occasionally used this method to 
target agents for file audits on an ad hoc basis. 

• The Office also does not determine the cost 
per property rights case. It informed us that, 
because the cost of using agents in property 
rights cases is relatively low, a lump-sum 
forecast of $400,000 for all cases is budgeted 
every year. 

• The Office has no time-docketing system in 
place to track the amount of time in-house 
legal and clinical staff spend on each case. As 
a result, it cannot do a comparison of handling 
a case in-house versus with panel lawyers to 
determine what is more cost-effective and effi- 
cient for different types of cases. It could also 
only estimate the time personal rights staff 
spent on supervision and quality assurance 
activities for panel agents, which it estimated 
took 60% to 80% of staff's time. The Office 
informed us that it was considering time- 
docketing for its new computerized case man- 
agement system, CHILD, which is discussed 
later in this report, but this function was not 
part of the project at the time of our audit. 

Co-operative Arrangements with Similar 
Organizations 

The Office has not formally examined opportunities 
for sharing costs and resources and co-operating 
with organizations that provide similar legal or sup- 
port services. For example, the Office of the Public 



Guardian and Trustee (OPGT) acts as Litigation 
Guardian for mentally incapable adults, and Legal 
Aid Ontario provides legal representation to adults, 
primarily using panel lawyers. Making use of such 
co-operative arrangements could help reduce 
overhead expenses and build capacity and might 
enable the Office to deliver its programs more cost- 
effectively. Approximately 25 of the Office's 85 staff 
perform administrative duties, and legal and clin- 
ical staff conduct training, and hire and supervise 
panel agents, in addition to working on cases. 

We identified opportunities for co-operative 
arrangements that include training of staff and 
panel agents, quality assurance programs for 
services provided, and the empanelment process 
used to select panel agents. We were informed that 
the Office has worked with Legal Aid Ontario in 
the past to offer training sessions to panel agents 
in conjunction with duty counsel. Also, OPGT and 
the Office provide similar services in property rights 
(for example, both may act as Litigation Guardian 
for adults and children, respectively), yet the Office 
has not explored opportunities to share resources 
with the OPGT. However, we did note that there 
is a protocol between the OPGT and the Office for 
cases where there may be potential duplication of 
services. For example, in an estate case where there 
is a child beneficiary and a mentally incapable adult 
beneficiary, and both the Office and the OPGT have 
taken the same position, only one office brings the 
case forward, so as to reduce duplication and costs. 



RECOMMENDATION 4 



To ensure that it has adequate management 
information on costs for services to enable it to 
more accurately assess the efficiency of both 
in-house staff and panel agents over time, the 
Office of the Children's Lawyer should collect 
information on the actual costs of completing 
its different types of cases and other activities. It 
should also explore opportunities for reducing 
its costs or enhancing its administrative capacity 
by collaborating with Ontario Public Sector 
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organizations that do similar legal work in 
areas like property rights and in fields such as 
training, quality assurance, and empanelment 
processes. 



OFFICE RESPONSE 



The Office is making changes to improve its 
case-cost information and its financial forecast- 
ing. A case-cost forecasting model has been 
developed to better analyze and predict total 
existing and new case commitments and costs. 
It provides more case information data and 
analysis on the average life cycle of a case, 
average and total case costs, and case type and 
category, including small, medium, and large 
total-dollar value. 

The Office will continue to implement 
measures to better monitor and control its 
total operations expenditures to ensure that it 
provides the most cost-efficient services within 
its budget allocation. The Office follows the 
OPS-wide Results-based Planning process and 
forecasts program expenditures in comparison 
to budget each month. Total expenditures have 
been limited to a 2.2% increase over the three 
fiscal years ending March 31, 2011. 

The Office will explore additional opportun- 
ities for co-operation with OPS organizations 
that provide legal services, including the Office 
of the Public Guardian and Trustee. 



INFORMATION MANAGEMENT SYSTEMS 

In 2002, the Office developed and implemented a 
computerized case management system called Case 
Track for its personal and property rights cases. 
Management also relies on this system to provide 
timely, accurate, and relevant information for 
decision-making purposes. 

In 2003, an internal audit followed by two 
subsequent systems evaluations raised serious 
concerns about Case Track's viability. The 2003 



internal audit report noted that the Office had no 
integrated case management system to effectively 
monitor and manage cases, nor were there suf- 
ficient access controls within Case Track to reduce 
the risk of unauthorized changes to case informa- 
tion, which may have been one of the reasons for 
unreliable and inconsistent data in the database. 
We found these issues still outstanding at the time 
of our audit. For example, Case Track was unable 
to track activity or status updates throughout a case 
and could capture information only at the opening 
and closing stages of a file. Statistics on cases sit- 
ting at the Intake and Accounts and Referrals units 
are gathered manually because it is not possible 
to record this information in Case Track. We also 
noted many instances of erroneous or missing data 
in Case Track, including almost 300 child protec- 
tion cases, which are mandatory for the Office to 
accept, that were recorded incorrectly as having 
been refused. 

A business technology solutions consultant 
the Office engaged to review its systems in 2007 
reported that Case Track was meeting only 25% of 
the Office's functional requirements and that the 
system's design made subsequent maintenance or 
enhancements prohibitively expensive. The consult- 
ant recommended replacing the Case Track system 
even though it was only five years old at the time. 
In response to these findings, the Office has been 
working with the Ministry's Justice Technology 
Services (JTS) to design and implement a new case 
management system called Children Information 
and Legal Database (CHILD) to address the cur- 
rent system's deficiencies. The total cost for this 
project was budgeted at $3.8 million at the time of 
our audit, with approximately $1.4 million having 
already been spent. The Office informed us that it 
expected CHILD to become operational in Novem- 
ber 2011. As of the end of our fieldwork, we found 
that the project was being managed according to 
the Management Board of Cabinet Information 
and Information Technology Directive and the OPS 
Integrated Project Management Framework and 
Methodology. 
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We understand that the system was designed in 
consultation with all of the Office's departments to 
ensure that it meets key business and user needs. 
However, we were informed by the project's team 
members that the new system will still meet only 
about 75% of the Office's business requirements. 
There was no documentation to support this 
informal assessment or what requirements consti- 
tuted the missing 25% but, for example, tracking 
the time Office staff spends on each case (time- 
docketing) was not within the scope of the new sys- 
tem's design, although this was initially identified 
as a business need for approving the CHILD system 
and the absence of time-docketing functionality in 
the Office's current system was noted as a deficiency 
in the 2007 business technology consultant's report. 



RECOMMENDATION 5 



To ensure that the new case management 
information system — Children Information and 
Legal Database (CHILD) — being developed will 
resolve deficiencies in the system it is replacing 
and meet current business and user require- 
ments, the Office of the Children's Lawyer, in 
conjunction with Justice Technology Services 
(JTS) project managers, should prepare an 
interim report for senior management compar- 
ing the deficiencies of the existing system to 
the intended functionality of the new system 
and identify any expected gaps or limitations in 
CHILD'S design. The interim report should also 
address how the new system will improve safe- 
guards for confidential information and improve 
data integrity and case file management and 
controls. 



The Office's new case management system, 
CHILD, has been designed to resolve many of 
the information technology, information man- 
agement, and process deficiencies identified in 
this audit. The system has been developed to 



meet 100% of the documented business require- 
ments signed off on by the Office- JTS Project 
Team and governance structure. The system has 
also been built using an iterative design meth- 
odology that will ensure that the application 
functionality aligns with business needs and 
process improvements. Important additional 
functionality, such as time-docketing, is planned 
as part of a future phase of the project. 

The Office and JTS are preparing an interim 
report that includes a gap analysis to describe 
how the new application will address existing 
system deficiencies. The project and governance 
teams are confident that the new system will 
meet the required levels of confidentiality and 
provide for improved data integrity and systems 
controls. 



TRANSITION TO ADULTHOOD 

We noted that there is no formal protocol in place 
to assist children turning 18 who have been repre- 
sented by the Office. Once a mentally competent 
child turns 18, the Office ends its involvement in 
any of his or her legal matters because it does not 
have the legal authority to act on behalf of adults. 
Children are notified in writing of the termination 
of the Office's involvement and are advised to 
retain their own counsel if they wish to continue 
to pursue a legal matter, such as a pending civil 
lawsuit. Without continuity of service or any type 
of planned transition or offer of support services, 
there is a risk that a child's interests will not be 
adequately protected after he or she turns 18. We 
acknowledge that there are legal limitations on the 
Office's further involvement when the child turns 
18 and is legally considered an adult capable of 
making informed decisions. However, the children 
to whom the Office provides property rights servi- 
ces typically may not have parental or other support 
or may have a legal conflict with family members, 
and may become responsible for complex estate 
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and injury cases involving significant financial 
matters. They also may not be able to afford further 
legal representation or qualify for Legal Aid Ontario 
support. In many cases, they may lack both the 
maturity and the experience to know what to do 
when they receive such a "now-in-your-hands" let- 
ter from the Office. 

The Office advised us that it does have informal 
arrangements in place with the OPGT for the transi- 
tion of minors who may be mentally incapable and 
are turning 18. The OPGT conducts an investigation 
when it receives information from the Office that a 
child turning 18 may be incapable and therefore at 
risk of suffering serious financial or personal harm 
and no alternative solution is available. This inves- 
tigation may result in the OPGT asking the court 
for permission to make decisions on the person's 
behalf. The Office informed us that it is currently 
working with the OPGT to develop a standard letter 
to be sent to the child and other affected parties in 
cases where a transition to the OPGT after the child 
turns 18 may be necessary. 



RECOMMENDATION 6 



To help ensure that children's interests continue 
to be adequately protected when they turn 
18 and no longer qualify for the legal services 
offered by the Office of the Children's Lawyer 
(Office), the Office should establish processes 
that include developing and communicating 
transition plans for each child, including refer- 
rals to appropriate support services. 

| OFFICE RESPONSE 

The Office recognizes the importance of provid- 
ing youth who have been represented by the 
Office and who turn 18 during the course of 
litigation with information to help them assume 
the responsibility to carry on the litigation. The 
Office will consider appropriate additional ways 
to assist in the transition. 



QUALITY ASSURANCE AND TRAINING 
PROGRAMS 

We found that the Office had established quality 
assurance processes and training programs to help 
ensure that legal and clinical investigative services 
were being consistently and competently delivered 
to children by qualified service providers. 

Performance Evaluations and File Reviews 

As previously mentioned, we were informed that 
personal rights staff lawyers and clinical investiga- 
tors estimate that they spend from 60% to 80% 
of their time supervising panel agents to ensure 
that they provide timely and quality services. The 
rest of their time is spent working on cases of their 
own, participating in committees, and planning 
for agent training. The performance of new panel 
agents is reviewed 18 months into the three-year 
empanelment period to determine whether the 
agent should remain on the panel, remain empan- 
elled on conditional status, or be removed from 
the panel. A similar review is conducted at the end 
of the empanelment period for those agents seek- 
ing re-empanelment. These reviews consist of an 
evaluation of an agent's performance based on cri- 
teria such as legal or clinical skills, compliance with 
Office policies and procedures, case management, 
and general administrative skills. 

The individual case files of all in-house staff and 
panel agents are also reviewed on a regular basis. 
Regional supervisors conduct these file reviews once 
every four months for new and conditional legal 
agents, once every six months for all other legal 
agents and in-house legal counsel, and quarterly 
for all clinical agents. Prior to the file review, the 
panel agent or in-house member of staff is required 
to submit a reporting letter or status review, which 
consists of a brief summary of the case, the work 
they have performed to date, and the work yet to 
be done. The supervisor then evaluates the quality 
of the agent's work against various criteria, such as 
the number of times the agent met with the child, 
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whether the position taken was appropriate, and 
whether sufficient information was gathered from 
collateral sources to support the position. Clinical 
agents are also required to submit their completed 
Children's Lawyer Report for approval by their 
supervisor and the Manager of Clinical Services 
before the final report is submitted to the court. 

Any concerns identified with the work of a panel 
agent are discussed with the agent; if the work has 
been unsatisfactory he or she may be put on a pro- 
bation period or removed from the panel. The Case 
Track system includes a reminder system to help 
ensure that supervisors complete file reviews and 
monitor agents at the required intervals. 

We reviewed adherence to the Office's estab- 
lished quality assurance processes and generally 
found that staff were meeting set timelines for 
performance evaluation and file reviews and iden- 
tifying significant areas of concern. As of Febru- 
ary 2011, 17 of 345 panel lawyers had been placed 
on conditional status. 

Training 

Upon empanelment, new agents are required to 
attend a one-day orientation where they are trained 
in Office policies and procedures and learn gener- 
ally how to conduct the various types of cases they 
will be assigned. In addition, the Office has training 
sessions in professional and administrative matters 
for legal and clinical agents twice a year. The Office 
decides on the type of training to be provided at 
these sessions through informal discussions with 
supervisors and senior management. 

We noted that there was no documentation or 
formal training plans targeting specific competen- 
cies needed by lawyers and clinical investigators for 
the type of work they perform. As well, the Office 
did not consistently record which agents had taken 
which training courses. The Office also does not 
offer makeup sessions for agents who have missed a 
training session. 

The most recent agreement, for the 2009-2012 
empanelment period, requires clinical agents to 



provide proof that they have completed a minimum 
of 21 hours of continuing education per year. How- 
ever, we learned in discussions with Office staff 
that they were not aware of this requirement. There 
was no documentation in any of the clinical agents' 
personnel files we sampled to show that the agents 
had completed the minimum required 21 hours 
of continuing education per year or that this had 
been assessed in their most recent quality assurance 
reviews. In-house clinical investigators are also 
expected to maintain 21 professional development 
hours per year through reading and attendance 
at seminars and conferences. We noted that the 
Office keeps track of the seminars attended by each 
investigator per year, but does not note the number 
of hours they have completed. 



RECOMMENDATION 7 



To ensure that it is reaping the full benefits of 
in-house training and continuing education 
requirements for its panel agents and its own 
staff, the Office of the Children's Lawyer should 
better document attendance at training and 
professional development activities so that such 
activities can be considered in its panel agents' 
and staff performance evaluations. 

| OFFICE RESPONSE 

The Office is committed to ensuring that it con- 
tinues to provide opportunities for training and 
development to its staff and panel agents: 

• The Office will more accurately document 
the attendance of panel agents at training 
sessions offered by the Office, and the time 
spent, and consider this in its agents' per- 
formance reviews. 

• The Office will more accurately document 
staff attendance and time spent at continu- 
ing education and professional development 
programs, and consider this information as 
part of performance management and learn- 
ing development plans. 
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MEASURING PERFORMANCE 

Performance Measures and Reporting 

Over the last several years, the Office has estab- 
lished and used 12 performance measures to assess 
program performance. These include measures of 
the timeliness with which services were delivered 
and of the results achieved. 

Three of the 12 measures are considered critical 
measures of success by the Office and are reported 
to the Ministry through the annual Results-based 
Planning process. These performance measures 
help report on the Office's success in assisting in 
getting cases resolved or settled without going 
to trial, and are appropriately outcome-based. 
However, all involve the Office evaluating its own 
success. For example, the measure that reports the 
percentage of cases where the Office's involvement 
assisted in resolving the matter is based on the legal 
agent's or in-house counsel's own assessment of his 
or her success in resolving the case, with no input 
from external stakeholders. Furthermore, there are 
no documented criteria against which this assess- 
ment is made. We also found that this measure 
needed to be more clearly defined, because it could 
be misleading — it claims to report on the Office's 
achievement on all cases, but the information used 
pertains only to custody and access cases, which 
represent only 26% of the Office's total cases. 

Two measures pertain to the legislative require- 
ment under the Courts of Justice Act that the Office 
serve and file a Children's Lawyer Report to the 
court within 90 days of serving notice to the parties 
of an investigation: the number of reports that meet 
the 90-day requirement and the number of reports 
that were completed within 120 days. However, as 
mentioned earlier, in the last five years, fewer than 
20% of the assigned reports were completed within 
the 90-day time frame. 

During the 2010/11 fiscal year, more than 
80% of all child protection cases received by the 
Office were handled outside of the Toronto area. 
Before April 2010, child protection cases outside of 
Toronto were assigned to the Office's panel agents 



by Legal Aid Ontario. This arrangement has since 
ended, and the Office now assigns all child protec- 
tion cases across the province. The Office has a 
performance measure in place that all child protec- 
tion cases in Toronto are to be assigned within five 
working days, yet it imposed no similar measure 
for assignment of child protection cases outside of 
Toronto. 

The Office prepares an annual review report for 
distribution to stakeholders such as Children's Aid 
Societies, the Ontario Bar Association, the Ontario 
and Superior Court Justices, and Legal Aid Ontario. 
This annual review provides only background infor- 
mation on the Office, a breakdown of expenditures 
for the year by department, the number of cases 
assigned in the year, and information on the Office 
staff's community involvement. In our view, the 
Office's annual report would be more informative 
and relevant to its stakeholders if it contained more 
useful and objective information on the Office's 
performance compared with its performance targets 
and if it were posted on the Ministry's public website. 

Consultation with Stakeholders 

Although the Office consults with panel members 
on its policies and procedures, there are no formal 
consultation processes in place with other key 
stakeholders — such as the child clients, Children's 
Aid Societies, and other parties to court proceed- 
ings — to regularly obtain feedback on the Office's 
effectiveness and the degree to which it meets 
expectations. 

We noted that in previous years the Office has 
asked its panel lawyers and clinical investigators 
about any concerns they might have regarding the 
Office's practices, such as the level of supervision 
needed, timeliness of payments, training provided, 
and other administrative matters, and communi- 
cated the results to them. The Office conducted a 
consultation with panel lawyers in 2007, and most 
recently again in 2010. However, as of June 2011, it 
had not yet communicated the results of the 2010 
discussions to them. 



Office of the Children's Lawyer 



The Office also does not hold consultations with 
the children it serves to determine whether their 
needs have been met by the services that were 
provided. We noted a practice in Alberta where the 
Office of the Child and Youth Advocate initiated 
a client feedback process to hear about children's 
expectations of their legal representative and how 
their experiences compared to those expectations. 
Questions included: 

Didyou understand your lawyer? 

Did your lawyer explain what was happening 

in court? 

Didyour lawyer listen to you? 

Did your lawyer tell the court what you 

wanted? 

Didyour lawyer answer your questions? 
Didyour lawyer explain what the judge's deci- 
sion means? 

Were you happy with the legal services you 
received fromyour lawyer? 

The Office informed us that the Children's 
Lawyer performs outreach and establishes dialogue 
with organizations doing work for families and 
children, as well as with the judiciary. From time 
to time, the Office's senior management also meet 
with provincial justices involved in family matters 
to discuss any concerns, but they did not have a 
record of the results of these meetings. 



RECOMMENDATION 8 



To help assess whether it is efficiently and 
effectively meeting the needs of its clients and 
stakeholders, the Office of the Children's Lawyer 
should continue to develop and report key 
performance indicators that are clearly defined 



and objectively measured, establish realistic 
targets, and measure and report on its success in 
meeting such targets. It should also implement a 
more formal process of obtaining periodic feed- 
back from stakeholders, such as its child clients 
and the judiciary. 

| OFFICE RESPONSI 

The Office acknowledges the importance of 
continuing to develop its key performance indi- 
cators (KPIs) in support of its core mandate and 
of measuring the results of internal processes 
and services provided to children and key stake- 
holder groups. 

The following key steps have been taken by 
the Office: 

• The Office's senior management team has 
already established a conceptual frame- 
work, based on best practices, and identi- 
fied a robust set of KPIs to drive results in 
alignment with key operating goals and 
strategies. The Office's new case manage- 
ment system will allow the Office to gather 
information about the services it provides in 
each region of the province. 

• The Office will continue to engage in direct 
outreach to key stakeholders to improve the 
information exchange with the Office. 

• The Office is developing a youth engagement 
strategy that will include dialogue with 
youth about the Office and its services. 

• The Office is committed to communicating 
more regularly with the public about the 
Office and what it does for the children of 
Ontario. 



Chapter 3 



Sectioi 

3.11 Ontario Trillium 
Foundation 



Background 



The Ontario Trillium Foundation (Foundation) was 
established in 1982 as an agency of the Ontario gov- 
ernment. Its mission is to build "healthy and vibrant 
communities throughout Ontario by strengthening 
the capacity of the voluntary sector, through invest- 
ments in community-based initiatives." 

It does this by distributing grants — about 1,500 
of them, worth more than $110 million, in the 
2010/11 fiscal year — to not-for-profit and charit- 
able organizations working in the areas of human 
and social services, arts and culture, environment, 
and sports and recreation. Most of the grant money 
goes to pay the salaries and wages of people work- 
ing in these organizations. 

The Foundation operates under the terms of a 
memorandum of understanding (MOU) with the 
Ministry of Tourism and Culture (Ministry) that is 
updated every five years, most recently in 2009. 
The MOU defines the Foundation's mandate and 
relationship with the Ministry regarding oper- 
ations, accountability, finances, administration, and 
reporting. 

The agency has a volunteer board of directors 
and approximately 120 full-time staff located at 
its Toronto head office and in 16 regional offices 
across the province. In addition, more than 300 



volunteers maybe named to grant-review teams 
across the province — there are 18 to 24 volun- 
teers on each team — to vote on which projects or 
organizations should be funded. At the time of our 
audit, 38% of grant-review team positions were 
vacant. The volunteer members of the board and of 
the review teams are appointed by the Lieutenant 
Governor-in-Council on the recommendation of 
the Minister of Tourism and Culture. 

From 1982 to 1999, the Foundation operated 
on approximately $16 million a year in Ministry 
funding drawn from provincial lottery revenues, 
and provided assistance only to groups involved 
in human and social services. In 1999, the Foun- 
dation's mandate was significantly expanded to 
include groups working in arts and culture, the 
environment, and sports and recreation. At that 
time, Foundation funding also increased to approxi- 
mately $100 million a year, drawn largely from 
charity-casino revenues. Since 2007, funding has 
come from general provincial revenues. 

In the 2010/11 fiscal year, the Foundation 
received total funding of about $124 million. About 
$111 million was paid out in grants to charitable 
and not-for-profit organizations, and the remainder 
covered program administration. The same year, 
the Ministry also committed an additional $50 mil- 
lion for a two-year capital-funding program with 
a focus on organizations serving culturally diverse 
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communities. Approximately $3.5 million in grants criteria so we were unable to reduce the extent of 
had been approved under this new program by the our audit work as a result, 
end of the last fiscal year. 




The objective of our audit was to assess whether 
adequate policies and procedures were in place at 
the Ontario Trillium Foundation (Foundation) to 
ensure that: 

• approved grants were consistent with the 
Foundation's mandate, in amounts that were 
commensurate with the value of the goods 
and services provided by the grant recipients, 
and that they were spent for their intended 
purpose; and 

• costs were incurred and managed with due 
regard for economy and efficiency, and the 
effectiveness of the Foundation was appropri- 
ately evaluated and reported on. 

Prior to our fieldwork, we identified criteria 
to be used to address our audit objectives. Senior 
management at the Foundation reviewed and 
agreed to these criteria. 

Our audit included a review of the Foundation's 
administrative and operational policies and pro- 
cedures. We also talked to selected staff members 
at eight locations — head office and seven regional 
offices in Toronto, Waterloo, Barrie, Kingston, 
Peterborough, Sudbury, and North Bay — and 
interviewed the chair of the board of directors. We 
reviewed and assessed pertinent grant, financial, 
and operational information, along with a sample 
of individual grant files. We visited 29 organiza- 
tions that received grants from the seven regional 
offices we visited. 

The Internal Audit Division of the Ministry of 
Tourism and Culture had not conducted any recent 
audits of the Foundation's operations. We did 
review reports from an individual contracted by the 
Foundation to conduct individual grant reviews. 
However, these reviews did not relate to our audit 



A primary responsibility of the Ontario Trillium 
Foundation (Foundation) is to ensure it gives out 
its annual allocation of more than $100 million 
each year to community not-for-profit and charit- 
able organizations. A wide range of projects can be 
funded, as long as they support the local commun- 
ity and relate to social services, arts and culture, the 
environment, and sports and recreation activities. 
For instance, grants can range from funding a light 
conservation project to reduce light pollution in 
the Bruce Peninsula, to supporting carbon-neutral 
farming, to developing employment skills for low- 
income newcomers, to strengthening the leadership 
skills of First Nations women. With respect to the 
question of value for money received for each grant, 
we acknowledge that this may well be in the "eye of 
the beholder" and that it is within this context that 
the Foundation operates. 

We found that the Foundation does ensure that 
all grants have a community-based focus. And while 
it has a well-defined grant application and review 
process for deciding which applicants receive 
grants, we noted that the underlying process and 
resulting documentation often did not demonstrate 
that the most worthy projects were funded in rea- 
sonable amounts because there was little evidence 
that the Foundation: 

• objectively compared the relative merits of 
different proposals to ensure the most worthy 
projects were supported; 

• adequately assessed the reasonableness of the 
grant amounts requested and approved; and 

• effectively monitored and assessed spending 
by recipients or the results they reported. 

In addition, many of the grant recipients we 
visited could not substantiate the expenditure and 
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community organizations. Nonetheless, we 
appreciate the Auditor General's recommenda- 
tions for stronger documentation, robust per- 
formance measures, and enhanced monitoring. 

The Foundation has introduced a new on- 
line grants-management system, developed 
and tested in the 2008/09 fiscal year, and fully 
implemented in March 2010. We are confident 
that the built-in controls are addressing many of 
the Auditor General's recommendations relating 
to the grant review and approval process and 
standardized documentation. 

The Foundation is committed to further 
enhancing its impact across the province, and 
welcomes the recommendations of the Auditor 
General to assist in its continuous improvement. 



performance information they reported to the 
Foundation. 

While its website is comprehensive and informa- 
tive, we believe the Foundation could do more 
to inform all community organizations of the 
availability of grants and the application process. 
It could, for example, consider advertising periodic- 
ally in local and ethnic-community newspapers. 

Although the Foundation's administrative 
expenditures were relatively modest compared to 
most other government agencies that we have aud- 
ited, it nevertheless needs to tighten up its admin- 
istrative procedures to ensure it complies with the 
government's procurement and employee-expense 
guidelines. 

| OVERALL FOUNDATION RESPONSE 

The Ontario Trillium Foundation (Foundation) 
appreciates the Auditor General's recommenda- 
tions. We acknowledge too the observations 
that the Foundation has a well-defined grant 
application and review process, and that there is 
an institutional mindset that places emphasis on 
keeping costs to a minimum. 

With 16 regional offices and approximately 
1,500 grants each year to not-for-profit organ- 
izations across Ontario, the Foundation is 
committed to building healthy and vibrant com- 
munities by serving the voluntary sector in all its 
diversity: large organizations and small, urban 
and rural communities, in every region of the 
province. The Foundation's grantee organiza- 
tions are all volunteer-led — and in many cases 
entirely volunteer-run — building communities 
with enthusiasm and often with very limited 
resources. 

Up to 300 community volunteers can be 
engaged in the Foundation's grant-making 
decisions, bringing their community experience 
to supplement the research and analysis of the 
professional staff. Effective community-building 
may not always fit a template, especially given 
the limited staff and resources of many small 



Detailed Audit Observations 



GRANT PROGRAM OVERVIEW 

The Ontario Trillium Foundation (Foundation), 
as outlined in the memorandum of understanding 
with the Ministry of Tourism and Culture (Min- 
istry), has a mandate to "provide funds in a fair and 
cost-efficient manner with community involvement 
in decision-making, and by way of supplementing 
rather than replacing regular sources of income, 
to eligible charitable and not-for-profit organ- 
izations in Ontario [...] to help finance through 
time-limited, results-oriented grants, programs 
undertaken by such organizations; and to help 
finance initiatives that increase organizational and/ 
or community capacity and self-reliance." 

The kinds of groups and projects that have 
received funding include small theatre companies, 
rural development initiatives, urban school food 
projects, multicultural festivals, cultural counsel- 
ling and support organizations, amateur sports and 
recreational associations, and local environmental 
initiatives. 



Ontario Trillium Foundation 



The Foundation has three funding programs, as 
follows: 

• The Community program, which receives 
approximately $83 million, or 77% of 
available grant money, covers activities in 
Ontario's 16 individual regions. Organizations 
can get up to $375,000 over five years under 
this program, including $75,000 per year for 
operating expenses and a total of $150,000 
for capital items such as building renovations 
or equipment purchases. Over the last four 
years, the Foundation provided approximately 
1,360 such grants each year with an average 
value of about $60,000 each. 

• The Province-wide program, which receives 
approximately $21 million, or 19% of the 
available grant money, covers either activities 
with a province-wide impact or those tak- 
ing place in at least three regions (two in 

the north) . Organizations may receive up to 
$1.25 million over five years, including up to 
$250,000 per year for operating expenses and 
a total of $150,000 for capital items. Over the 
last four years, the Foundation awarded about 
110 such grants each year with an average 
value of about $180,000 each. 

• The Future Fund, which receives approxi- 
mately $4 million, or 4% of the available grant 
money, covers projects that create significant 
and sustainable change in a specific area using 
distinct and innovative approaches. The focus 
for the 2010/11 fiscal year, for example, was 
on creating economic opportunities for youth 
and building skills for the green economy. 
About 10 such projects are funded each year 
with an average value of $400,000 each. 

The Foundation's goal is to dispense the entire 
annual grant funding that it receives from the 
government. It allocates funds for the Community 
program to its 16 regions on a per-capita basis. In 
the 2010/11 fiscal year, per-capita funding, based 
on census information, was approximately $6.64. 
As the different regions had varying population 



levels, total annual funding to each ranged from a 
low of $ 1.4 million to a high of $ 16.6 million. 

We found that total funding requests relative 
to the amount of funding provided varied signifi- 
cantly, both within and between regions. Approval 
rates for community grants ranged from a low of 
23% of funds requested in all applications in one 
region to a high of 58% in another during the 
2009/10 fiscal year, as illustrated in Figure 1. 

Allocation of funding to the regions on a per- 
capita basis facilitates equitable access to grant 
funds throughout Ontario, but it is not intended 
to ensure that the most worthy projects across the 
province are actually funded, as outlined in the sec- 
tion on the grant review and approval process. 

GRANT PROMOTION 

The Foundation has three main ways of promoting 
the availability of grants to the public: 

• Website and social media: The Foundation 
maintains a comprehensive and informative 
website that outlines, among other things, 
its mission, its granting programs, and how 
a group can apply for funding. Social media 
approaches — including Twitter, Facebook, 
and blogs — were introduced in 2011. 

• Media and announcements: The Foundation 
participates in more than 700 ceremonies 
held by grant recipients each year; these 
events generate more than 4,500 articles each 
year in print and broadcast media. 

• Solicitation and word of mouth: Foundation 
staff proactively seek out and communicate 
with organizations about potential projects 
that they think could meet the criteria for 
grant funding. In addition, board and grant- 
review team volunteers spread the word 

in communities and among community 

organizations. 
However, the Foundation, which is required 
to "provide funds in a fair... manner," does not 
publicly advertise the availability of grants in any 
formal way; it buys no ads, for example, in print or 
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Figure 1: Approval and Grant Allocation Rates for Trillium Community Program Grants by Region, 
2007/08-2009/10 

Source of data: Ontario Trillium Foundation 





Approval Rate (% of Approved Grant Funding 
Compared to Total Funds) 


Three-year 
Average 


1 Region/Catchment 


2007/08 


2008/09 


2009/10 


($ million) 


Algoma, Cochrane, Manitoulin, Sudbury 


35 


40 


34 


3.22 


Champlain 


30 


33 


29 


7.00 


Durham, Haliburton, Kawartha, Pine Ridge 


59 


44 


45 


6.15 


Essex, Kent, Lambton 


33 


33 


34 


4.35 


Grand River 


38 


36 


33 


1.91 


Grey, Bruce, Huron, Perth 


66 


49 


40 


2.42 


Halton-Peel 


43 


46 


34 


9.11 


Hamilton 


33 


25 


27 


2.98 


Muskoka, Nipissing, Parry Sound, Timiskaming 


50 


58 


33 


1.18 


Niagara 


45 


43 


38 


3.15 


Northwestern 


32 


28 


58 


1.95 


Quinte, Kingston, Rideau 


46 


42 


35 


3.81 


Simcoe-York 


41 


39 


34 


8.10 


Thames-Valley 


36 


32 


23 


4.15 


Toronto 


30 


31 


28 


15.06 


Waterloo, Wellington, Dufferin 


37 


31 


35 


4.77 



broadcast media. As a result, there is little assur- 
ance that all eligible organizations even know that 
there is an Ontario Trillium Foundation and that 
grant money may be available to them. Unless 
someone in an organization discovers the avail- 
ability of grants and then visits the website, that 
organization misses out on an opportunity. 

We found evidence that other granting bod- 
ies use print or broadcast media to publicize the 
availability of grants. For example, grants available 
under the New Horizons for Seniors Program of 
Human Resources and Skills Development Canada 
are advertised in 50 newspapers across the prov- 
ince. In addition, York Region advertised funding 
available through the New Agency Development 
Fund in local print media in spring 2011. 

The solicitation of applications by staff and the 
Foundation's volunteers, including grant-review 
team members, also raises the issue of potential 
conflict of interest as the same people who invite 
certain groups to apply for grants, or who tell them 



about the program, later review those applications 
and determine who gets funding. 

We noted, for example, that two board members 
also own consulting businesses that provide service 
to the not-for-profit sector. We examined one of the 
businesses and found that of the 11 projects listed 
on its website, six had received Foundation grants 
during the time the owner was on the Foundation 
board. One of the grants included money for con- 
sulting services bought from the board member's 
business. Although we understand that the consult- 
ant's business got the contract only after a formal 
bidding process, arrangements of this nature run 
the risk of being viewed as a conflict of interest. 



RECOMMENDATION 1 



To ensure that all qualified organizations get 
a fair chance to learn about and apply for its 
grants, the Ontario Trillium Foundation should: 



Ontario Trillium Foundation 




• publicly advertise information about its 
grants, application deadlines, and its web- 
site; and 

• investigate ways to reduce or eliminate 
perceived or real conflicts of interest by 
ensuring that the people who encourage 
organizations to apply for grants are not the 
ones who subsequently help select which 
applications will be funded. 

| FOUNDATION RESPONSE 

The Foundation recognizes the value of con- 
tinually enhancing the level of awareness of its 
programs. We have received more than 16,000 
grant applications in the past five years, and 
annually receive $3 to $4 in requests for every 
$1 available. Building upon our comprehensive 
website and cost-efficient media strategy, the 
Foundation will investigate and institute new 
forms of generating publicity about its grant 
programs. 

While the Foundation's conflict-of-interest 
policies have served it well in the past 29 years, 
we agree with the Auditor General's recom- 
mendation that there is a need for continued 
enhancements, and we will investigate ways to 
further reduce or eliminate perceived or real 
conflicts of interest. 



GRANT REVI EW AN D APPROVAL 
PROCESS 

The Foundation puts grant applications for all three 
funding programs through a standard review pro- 
cess as follows: 

• Technical review: Applications are initially 
screened to ensure that they are complete and 
that applicants meet basic eligibility criteria, 
such as being either a not-for-profit or a 
charitable organization with a board of direc- 
tors. Proposed projects must include, among 
other requirements, an operating budget. 



Incomplete applications, and those deemed 
ineligible, are rejected in this phase. 

• First review: Foundation staff apply a 
15-question test to applications, rating them 
on points for each question. Totals are then to 
be used to rank projects. 

• Triage meeting: Regional staff and grant- 
review team members meet to vote on which 
applications to reject. The remaining ones 
move forward. 

• Additional research and analysis: Projects are 
further scrutinized by staff, who are supposed 
to conduct site visits for community projects 
seeking more than $100,000 and for province- 
wide projects asking for more than $500,000. 

• Proposal Assessment Summary Sheet (PASS): 
Information collected during the research 
phase is consolidated into the PASS, which 
also recommends whether the application 
should be approved or declined. 

• Final meeting: The grant-review team meets 
to recommend approval or rejection. If it opts 
to approve, the team also recommends how 
much funding the project should get. 

• Final approval: The Foundation's CEO or its 
Board approve or reject the proposal. We 
understand that projects making it all the way 
to this stage are rarely rejected. 

Given the Foundation's broad mandate, and 
the fact that it solicits many of the project propos- 
als it receives, it is unusual for an application to 
be rejected for falling outside the Foundation's 
mandate. 

With respect to the phases of the application 
review process, we found the following: 

• The technical review is an objective process 
that usually weeds out those applications that 
are ineligible or missing information. 

• Although regional offices are required to 
complete the 15-question first review for 
each application that passed the technical 
review, we found that many of the case files 
we reviewed contained no evidence that this 
had been done. Even when the 15-question 
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test was on file, it was improperly completed 
in half the cases we reviewed. 

• Five of the eight offices we visited did not use 
the total score from the first review to rank 
projects, as intended by the procedure. The 
three others generally used the process in the 
way that we would expect, but there were 
instances where, without explanation, lower- 
ranked projects advanced while higher-ranked 
ones did not. 

• Work conducted on applications following the 
first-review and triage-meeting phases, and 
the quality of documentation, varied signifi- 
cantly and in our view was often inadequate. 
In addition, we found in a sample of files that 
site visits, required for projects of a certain 
size, were either poorly documented or not 
done at all. 

• Regardless of whether the PASS document 
supports funding, it does not provide a viable 
basis for comparing one project to another. As 
a result, there was no comparative documenta- 
tion to indicate why some projects were rec- 
ommended for funding and others were not. 
This meant that there was a lack of documen- 
tation to demonstrate that the relative merits 
of proposals had been objectively compared. 

• At the final grant-review team meetings that 
we attended, there was little discussion and 
debate, and all of the recommended projects 
presented were approved. 



RECOMMENDATION 2 



To help ensure that grant decisions are objective 
and supportable, the Ontario Trillium Founda- 
tion should: 

• make sure each of its regional offices com- 
pletes the 15-point questionnaire and uses it 
to assess and prioritize grant applications; 

• develop consistent guidelines, policies, and 
procedures for staff and grant-review teams 
to follow when assessing grant applications, 
and make sure any required site visits are 
conducted; and 



• maintain documentation that provides a 
basis for comparing one project to another to 
clearly demonstrate why some projects were 
funded and others not. 



FOUNDATION RESPONSI 



The Foundation agrees with the Auditor General 
regarding the value of greater consistency of 
grant-making procedures and better documen- 
tation. This is particularly relevant in the con- 
text of our regional structure. This need was one 
of the driving factors behind the development 
and implementation of our new on-line grants- 
management system, developed and tested in 
the 2008/09 fiscal year, and fully implemented 
in March 2010. 

The new system is enforcing the standardiza- 
tion of consistent procedures and documenta- 
tion, and we will continue to actively monitor 
the success of this objective. We acknowledge 
the recommendation to better document the 
comparability of projects and we will work to 
review methods of doing this effectively. 



REASONABLENESS OF AMOUNTS 
APPROVED 

As the biggest component of many projects funded 
by the Foundation is salaries and fees, it is import- 
ant to assess the reasonableness of these proposed 
costs in applications. Our review of a sample of files 
for approved grants found that they often did not 
contain the information from applicants needed 
to assess this. Accordingly, we questioned how the 
Foundation was able to adequately assess the rea- 
sonableness of the grant amounts requested. Based 
on the available information, we were often unable 
to determine for ourselves whether the grant 
amounts were commensurate with the services to 
be provided because we could not assess either the 
reasonableness of the specific services or deliver- 
ables the organizations proposed to provide, or the 
work required to meet the objectives. 



Ontario Trillium Foundation 




Examples of projects where the reasonableness 
of funding amounts was not established include: 

• a grant of $120,000 to a community organiza- 
tion with one staff member over a two-year 
period to develop a strategic plan for itself; 

• $400,000 over 36 months to an organization 
to enable Ontario sports leaders and organiza- 
tions to collaborate, innovate, and better con- 
tribute to social and economic development in 
their communities; 

• a grant of $132,000 to an organization to 
deliver a training program on self-employment 
to newcomers to Canada; 

• $222,000 over three years for hiring at-risk 
individuals and starting up a community gar- 
den program; 

• $537,000 over three years to provide leader- 
ship programs to women in First Nations com- 
munities; and 

• $35,900 for a year to increase citizen aware- 
ness and reduce light pollution in the Bruce 
Peninsula. 

The Foundation also requires that grant recipi- 
ents obtain two quotes when buying items costing 
more than $1,000 (increased to $5,000 in March 
2010) . However, we found a number of cases where 
there was no evidence the grant recipients actually 
obtained the required competitive quotes. Some 
examples were: 

• a multicultural cinema club given $40,000 for 
camera equipment; 

• a soccer club awarded $34,000 for a new com- 
puter system; and 

• an environmental group funded for a 
$125,000 renovation. 



RECOMMENDATION 3 



To help ensure that grant amounts are reason- 
able and commensurate with the value of goods 
and services to be received, the Ontario Trillium 
Foundation should: 

• assess and adequately document the rea- 
sonableness of the specific services or deliv- 



erables organizations say they will provide 
with the money they are requesting; and 
• objectively assess the required work effort 
or other resources needed to meet the stated 
objectives of the grant application. 



The Foundation agrees with the Auditor General 
that the assessment of the reasonableness of 
grant amounts approved is an essential compon- 
ent of effective grant-making. With most organ- 
izations in Ontario's not-for-profit sector, there 
is a commendable culture of cost containment, 
thanks in large part to committed donors and 
volunteers in the sector. 

Grant-making decisions at the Foundation 
involve detailed discussions at various points 
in the multi-stage review and approval process. 
We agree that the assessment of reasonableness 
of costs needs to be better documented, so as 
to more clearly demonstrate the analysis done. 
The new on-line grants-management system, 
developed and tested in the 2008/09 fiscal 
year, and fully implemented in March 2010, 
provides an excellent platform for enhanced 
documentation. 



GRANT MONITORING 

Grant recipients are required to submit annual 
progress reports for the duration of the grant term 
and a final report within two months following 
project completion. These reports must include 
information comparing budget allocations to actual 
expenditures, as well as what was accomplished 
with the money they received. 

We found the process to be inadequate for 
ensuring that money was spent for the intended 
purposes. In particular: 

• Although groups are required to report 
back in summary form to the Foundation on 
spending, our review of a number of these 
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found that there was insufficient detail to 
assess the reasonableness of amounts spent or 
whether organizations were simply reporting 
the original budgeted amounts as the actual 
expenditures, without showing what actual 
expenditures were. 

• While grant recipients are expected to retain 
receipts and invoices for audit purposes, they 
are not required to submit them to the Foun- 
dation to substantiate the expenditures. We 
noted the Foundation rarely requests these 
documents to spot-check that the reported 
expenditures were actually incurred as 
reported by the recipient. 

• Grant recipients are not required to submit 
documentation to substantiate the perform- 
ance information that they provide in the 
progress reports and the final reports. 

• In the sample we reviewed, there was often 
inadequate evidence that Foundation staff 
questioned the progress and final reports sub- 
mitted by grant recipients. 

• Reports were often late — one-third of the 
progress reports in our sample were late by 
an average of four months and one-quarter of 
final reports were late by an average of seven 
months. 

• The Foundation requires no site visits by staff, 
even on a sample or risk-assessment basis, to 
assess what has been accomplished with Foun- 
dation funds, and site visits are rarely done. 

• The Foundation hires an outside contractor 
to review about 1% of completed grants and 
grant recipients each year. We found that 
these reviews are limited in scope because 
they focus on evaluating support for expendi- 
tures and do not include results achieved. In 
addition, the contractor makes no site visits 
and simply has supporting information sent to 
the office. 

Our site visits to a number of grant recipients 
found a number of instances where spending of 
grants was not adequately documented; where the 
amounts spent appeared excessive and were not 



supported by documentation; and where grant 
money was used for purposes other than those 
approved. For example: 

• More than half of the organizations that we 
visited did not have sufficient receipts avail- 
able to support the amounts they said they 
had spent. 

• In almost all cases the organizations could not 
provide evidence of the hours worked or what 
actual work was undertaken by people in the 
funded positions. 

• A grant of $73,000 was provided to an organ- 
ization for air quality tests, including $31,600 
for salary costs and $23,000 for equipment. 
We found little evidence of any work done — 
except a recording of eight hours of visits to 
two schools over the course of a year. In addi- 
tion, the air-testing equipment purchased with 
the grant could not be located during our visit. 

• Funds were provided to an organization to 
renovate a soup kitchen, including $26,000 for 
landscaping and $12,000 for steam-cleaning 
equipment. We found only $2,600 was spent 
on landscaping, and the steam-cleaning equip- 
ment was never purchased. Instead, the funds 
were spent on other renovations that were not 
approved. 

• A non-profit housing corporation received 
$48,000 to help integrate youth from its com- 
munity into the wider population. However, 
the grant predominantly supported a range of 
recreational activities, including makeup les- 
sons, and outings involving skiing, laser tag, 
and minigolf. 

• An organization received $5,000 to purchase 
transmitting equipment for a radio station but 
we could not locate the equipment during our 
visit, and the organization could not provide 
an invoice or other receipt to show it had ever 
been purchased. 

• Two organizations we visited had not spent 
all the grant funds they received, even though 
they said they had on their final reports. One 
of these organizations returned $6,600 from 
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a $75,000 grant more than a year after its 
final report was submitted while another kept 
the unspent $10,000 from an $81,000 grant 
because it was reported as spent. 



RECOMMENDATION 4 



The Ontario Trillium Foundation should 
strengthen its monitoring efforts to help ensure 
that funds are used for their intended purpose, 
and that reported purchases were actually 
made, by: 

• implementing periodic quality assurance 
reviews of grant files to ensure compliance 
with internal policies and requirements, and 
assessing the appropriateness of decisions 
made by granting staff; 

• expanding on the process undertaken by the 
contracted individual to include more thor- 
ough reviews of granting information; 

• requiring organizations to submit sufficiently 
detailed information to enable the Founda- 
tion to assess the reasonableness of the 
amounts spent; 

• conducting more audits of progress and final 
reports submitted by grant recipients; and 

• conducting site visits, where applicable, to 
see how grant money was spent. 

| FOUNDATION RESPONSE 

The monitoring of grants in a cost-efficient 
way is a challenge that all grant-makers face. 
The Foundation has always worked to achieve 
an optimal balance between maintaining cost- 
efficiency, while auditing and verifying a sample 
number of grant records each year. 

The Auditor General is recommending that 
additional resources be used to implement qual- 
ity assurance reviews, expand internal audit 
functions, request and review more grantee 
documentation, and conduct more site visits. The 
Foundation acknowledges and respects these rec- 
ommendations, and will investigate cost-efficient 
ways of expanding these procedures. 



PERFORMANCE MEASURES 

The Foundation has outlined for itself a set of 
performance measures intended to report on 
its performance as a granting organization and 
to determine whether grants met the intended 
outcomes. 

The performance evaluation measures include: 

• allocating a specific percentage of funds to 
each granting priority and sector (for example, 
human and social services is designated at 
30% to 50%, arts and culture at 20% to 50%, 
and sports and recreation at 20% to 50%); 

• a goal of dispensing 100% of a year's ministry- 
approved grant budget to recipients by the 
end of the fiscal year; 

• a goal of an average turnaround time for grant 
decisions of 120 days; and 

• maintenance of the Foundation's "high rank- 
ing" in terms of cost-effectiveness of average 
administration expenses for each grant. 

These criteria, intended to measure the Founda- 
tion's own performance, provide information that 
may be of interest to the Foundation and the public. 
However, they are not useful for actually assessing 
the Foundation's success in meeting its objective 
to fund worthy projects in the right amounts, or 
for identifying areas in its operations that need 
improvement. Two of the measures — allocation 
of funds and cost-effectiveness — are too broadly 
defined to yield meaningful assessments. 

The performance measures aimed at determining 
whether grants meet intended outcomes include: 

• the percentage of grant recipients that meet 
"all or some" of their program targets; 

• the value of additional leveraged contribu- 
tions for every dollar granted, with a goal of 
generating an additional $1.50 to $2 from the 
recipient for every $1 of grants; and 

• the percentage of grant recipients that provide 
recognition to the Foundation, with a goal of 
90%. 

The first two of these could be reasonable meas- 
ures of performance. However, the evaluation of 
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these measures is based on unverified information 
reported by grant recipients themselves. The value 
of additional leveraged contributions, for example, 
is based on estimates of such factors as volunteer 
hours reported by recipients in their final reports. 
In many of our visits to recipients, we could find 
little evidence to support the information reported 
to the Foundation. 



RECOMMENDATION 5 



To help assess whether the Ontario Trillium 
Foundation (Foundation) is meeting its stated 
objectives, and to help identify in a timely man- 
ner those areas needing improvements, the 
Foundation should: 

• establish meaningful operational indica- 
tors and realistic targets, and measure and 
publicly report on its success in meeting such 
targets; and 

• substantiate, at least on a sample basis, the 
information obtained from grant recipients 
that is used to evaluate success in meeting 
targets. 



| FOUNDATION RESPONSE 




The Foundation agrees with the Auditor General 
about the value of meaningful operational 
indicators and targets. Within the context of our 
multi-sector, highly diversified community and 
province-wide grant-making, we will continue 
to investigate and develop stronger operational 
indicators. This is a challenge that faces most 
grant-making foundations around the world, 
and the Foundation is committed to setting and 
maintaining high standards in this area. 

We agree with the Auditor General's recom- 
mendation that we substantiate, at least on a 
sample basis, the indicators of success that are 
communicated by our grantee organizations. 



GOODS AND SERVICES PROCUREMENT 

The Foundation's administrative expenditures total 
approximately $13 million a year, of which $9 mil- 
lion is for employee wages and benefits and $4 mil- 
lion for the acquisition of goods and services. We 
noted an institutional mindset that placed emphasis 
on keeping costs to a minimum. 

However, the Foundation is required to follow 
government procurement policies, which for con- 
sulting services require: 

• three competitive proposals for services up to 
$100,000; and 

• an open competition for bids through 
Ontario's electronic tendering system for ser- 
vices over $100,000. 

For the acquisition of all other goods and servi- 
ces, the policy requires: 

• three verbal quotes for anything valued 
between $5,000 and $25,000; 

• three written quotes for anything valued 
between $25,000 and $100,000; and 

• an open competition for bids through 
Ontario's electronic tendering system for any- 
thing over $100,000. 

We reviewed a sample of both types of contracts 
and found that half were single sourced and lacked 
adequate documentation to support the rationale 
for single sourcing. In addition, for a quarter of 
the contracts we reviewed, the appropriate level of 
approval was not documented. 

We also reviewed a sample of employee claims 
for travel, meal, and hospitality expenses, and for 
foundation-organized staff functions, and found 
that they frequently lacked the detailed information 
required to assess the reasonableness of the items 
and amounts claimed, as well as documentation to 
prove they were business-related expenses. 



RECOMMENDATION 6 



To help ensure that the Ontario Trillium Foun- 
dation (Foundation) follows the government's 
directives on the acquisition of goods and 
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services, as well as travel, meal, and hospitality 
expenses, the Foundation should reinforce with 
staff the need to comply with the directives, 
and consider having the Ministry of Finance's 
Internal Audit Division periodically review com- 
pliance and report the results of such reviews to 
the Foundation's Board. 



| FOUNDATION RESPONSI 



The Foundation is committed to following the 
government's directives in these areas, and has 
been strengthening its internal policies over the 
last few years. Consistent documentation of pro- 
curement decisions is also being strengthened. 

The overall modesty of our operating costs 
bears testament to our commitment to cost- 
efficiency. We appreciate the Auditor General's 
recommendation regarding periodic review by 
the Ministry of Finance's Internal Audit Div- 
ision. We would welcome this, and have initiated 
discussions with the Ministry in this regard. 



OTHER MATTER 

Conflict-of-interest Declarations 

Persons hired as staff and volunteer appointments 
to grant-review teams and the board of directors 
are required to sign conflict-of-interest declarations 
that they have read, understand, and agree to com- 
ply with the Foundation's conflict-of-interest policy. 
However, such individuals are not required to iden- 
tify people or organizations with whom they may 
have a potential conflict of interest, as is required by 
many other organizations. 

Given the nature of its grant program, the 
Foundation often recruits staff and volunteers 
from the same community as grant recipients, and, 
as a result, many already know individuals from 
organizations seeking grants. Accordingly, it is 
particularly important for the Foundation to ensure 
that new staff and volunteers identify potential con- 



flicts. Otherwise, the Foundation cannot effectively 
oversee and monitor potential conflicts of interest. 
However, we also found that there is no require- 
ment that individuals periodically update or renew 
their conflict-of-interest declarations, as required 
annually by many other organizations. We noted 
that the Foundation could not locate a few of the 
conflict-of-interest declarations that we asked to see 
for individuals who began working for the Founda- 
tion in the last three years. 



RECOMMENDATION 7 



To help ensure that its conflict-of-interest policy 
is effectively enforced, the Ontario Trillium 
Foundation should more effectively oversee and 
monitor compliance with its conflict-of-interest 
policy by staff, members of the board of direc- 
tors, and grant-review team members. It should 
also require them to update or renew their 
conflict-of-interest declarations annually, and 
include a listing of individuals and organiza- 
tions with whom they have a potential conflict 
of interest. 



OUNDATION RESPONSI 



The Foundation's volunteers and staff are all 
highly engaged members of their communities, 
participating actively in voluntary work and 
community-building. The Foundation agrees 
with the Auditor General regarding the value of 
effective and clear conflict-of-interest policies 
and practices. While our policies have served us 
very well over the years, we are committed to 
ongoing improvement. 

The Foundation has instituted the annual 
signing of conflict-of-interest declarations. We 
will investigate best practices in relation to the 
creation and maintenance of a list of organiza- 
tions with which individuals have a potential 
conflict of interest, as recommended by the 
Auditor General. 



Chapter 3 



Ministry of Training, Colleges and Universities 



Sectionl 

3.12 Private Career Colleges 




Background 



Private career colleges are independent organiza- 
tions that offer certificate and diploma programs to 
students in fields such as business, health services, 
information technology, and electronics. Private 
career colleges also cater to adults who need 
specific job skills to join the workforce or want 
to enhance their practical skills to become more 
competitive in the job market. There are about 470 
registered private career colleges in Ontario, with 
650 campuses and an estimated 60,000 students. 

Private career colleges are governed by the Pri- 
vate Career Colleges Act, 2005 (Act) which came into 
force on September 18, 2006. The Ministry of Train- 
ing, Colleges and Universities (Ministry) admin- 
isters the Act through its Private Career Colleges 
Branch. The Branch, headed by a director referred 
to in the Act as the Superintendent of private career 
colleges, has 30 staff including contract staff and 
spent almost $3 million in the 2010/11 fiscal year. 

Under the Act, institutions that provide instruc- 
tion in the skills and knowledge required to get a 
job in a particular vocation must be registered, and 
their vocational programs must be approved by the 
Superintendent. Private career colleges currently 
offer more than 5,000 programs excluding non- 
vocational programs and programs that are exempt 



from the Act, such as programs exclusively for 
youths and programs providing religious vocational 
training. Private career colleges must also comply 
with a number of other obligations, including those 
pertaining to program delivery, instructor qualifica- 
tions, and admission requirements. 

According to the Ministry, the Act is focused 
on student protection, and the Ministry's primary 
objective is to protect students and prospective stu- 
dents of private career colleges. These protections 
include the right to receive a refund of fees, access 
to a student complaint process, the right to receive 
transcripts for at least 25 years, and the opportun- 
ity to complete their training, at no additional cost, 
at another institution if their original private career 
college ceases operations. The costs for completing 
training under the latter circumstances are borne 
by the Training Completion Assurance Fund, which 
receives contributions from registered private 
career colleges to protect students in the event of 
a closure. The Ministry has significant powers to 
ensure that private career colleges comply with the 
Act and its regulations, including the ability to enter 
and inspect the premises of a registered private 
career college or an unregistered institution that 
ought to be registered. 

Although the Ministry does not fund private 
career colleges directly, the Ministry provides 
significant funding to the private career college 
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sector through its employment training and student 
assistance programs. Over the past three fiscal 
years (2007/08 through 2009/10), a total of almost 
$350 million was provided through the Ministry's 
Second Career and Skills Development programs 
to an annual average of 13,000 students to pay 
for their tuition to attend private career colleges. 
In addition, during the last three academic years, 
almost $200 million in provincial loans and grants 
were provided to an annual average of 9,500 pri- 
vate career college students through the Ministry's 
Ontario Student Assistance Program (OSAP). 



Audit Objective and Scop 



The objective of this audit was to assess whether 
the Ministry of Training, Colleges and Universi- 
ties had adequate procedures in place to meet its 
legislated responsibilities to protect existing and 
prospective students of private career colleges in 
Ontario and to measure and report on its effective- 
ness in doing so. 

Senior management reviewed and agreed to our 
audit objective and associated audit criteria. 

Our audit work was primarily conducted at 
the Ministry's Private Career Colleges Branch. We 
also contacted associations representing private 
career colleges in Ontario to obtain their views and 
conducted an independent survey of 500 recent 
graduates about their satisfaction with the training 
they received, their employment status, and their 
awareness of their rights under the Act. We visited 
a few campuses, but because our audit focused on 
ministry controls and procedures, we did not audit 
any private career colleges. 

In conducting our audit work, we reviewed 
relevant legislation, policies, and procedures, and 
met with appropriate staff of the Ministry. We 
also researched other jurisdictions. Our audit also 
included a review of related activities of the Min- 
istry's Internal Audit Services Branch. We reviewed 
the Branch's recent reports and considered its 



current and planned work, and any relevant issues 
identified when planning our work. 




The Ministry has recently undertaken several good 
initiatives to improve its oversight of private career 
colleges in Ontario and strengthen the protections 
for students. Nevertheless, further improvements 
are required to ensure compliance with the Act, 
its regulations, and ministry policies, and to bet- 
ter ensure that the Ministry's primary objective 
of protecting students is met. Some of our more 
significant observations are as follows: 

• Although several steps have been taken to 
identify and act on unregistered colleges, the 
Ministry could make better use of information 
it already has on hand to identify colleges that 
continue to operate illegally. For example, 
the Ministry does not routinely check that 
closed schools have not continued to operate 
without the necessary ministry approvals. 
We reviewed a sample of schools identified as 
being closed and found that a number of them 
appeared to be offering unapproved courses. 
We informed the Ministry, and in two of these 
cases it subsequently took enforcement action. 

• Although the Ministry collects and publishes 
performance information such as graduation 
rates and employment for public colleges, 
the Ministry does not collect similar informa- 
tion for private career colleges. Over 85% 

of the private career college graduates who 
responded to our survey agreed that such 
student outcome data would be useful. The 
Ministry used to collect this information for 
O SAP-approved private career colleges, but in 
2006 placed a temporary moratorium on the 
collection of this information to allow for a 
review of the process to take place. It has yet 
to re-establish this practice. 
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• According to the Act, in order to open a pri- 
vate career college, an applicant must satisfy 
the Ministry that it can be expected to be 
financially responsible and operate the college 
"in accordance with the law and with integrity 
and honesty." Although we found that the 
Ministry generally had the required registra- 
tion documentation on file, we had concerns 
about the adequacy of some of the procedures 
undertaken to assess this documentation such 
as a lack of verification through credit and 
reference checks. 

• Private career colleges are required to renew 
their registration annually. Although applica- 
tions for renewal must be accompanied by 
financial statements, and the Ministry intends 
to phase in a requirement that these state- 
ments be audited, the Ministry did not have 

a process in place for reviewing the financial 
statements submitted to determine if a col- 
lege's financial viability was in question. A 
private career college with significant losses, 
which the Ministry attributed to declining 
enrolment, that also appeared to be depend- 
ent on shareholder loans to meet its financial 
obligations had its registration renewed 
without evidence that its financial viability 
had been reviewed. The college subsequently 
closed, costing the Training Completion 
Assurance Fund over $800,000. The Ministry 
advised us that the college's inability to meet 
new regulatory requirements may also have 
contributed to its closure. 

• According to legislation, for a program to 
be approved, it must provide the skills and 
knowledge required to obtain employment in 
a prescribed vocation. In addition, legislation 
requires program applications to include an 
evaluation of the program by an individual 
who has expertise in the assessment of such 
programs. Our review of a sample of evalua- 
tions completed by third-party assessors 
revealed that there was no documented evi- 
dence that the Ministry had attempted to con- 



firm their credentials, although the Ministry 
informed us that it had begun to keep track of 
validated assessors. In addition, in most cases, 
neither the applicant nor the assessor had 
declared, as required, that they were not in a 
potential conflict of interest. We noted a case 
in which a conflict appeared to exist where a 
program assessor who previously had been 
employed by the college in question had also 
been involved in developing a curriculum for 
that college. 

• To continue to provide the skills and knowledge 
needed to obtain employment in a prescribed 
vocation, most approved programs need to 
adapt over time to meet market demands. Pro- 
grams approved by the Ministry subsequent 
to the proclamation of the new Act and one of 
its regulations, on September 18, 2006, can 
be approved for a maximum of five years, but 
approximately 40% of the 5,000 currently 
approved programs were approved before the 
proclamation of the current Act, and the Act 
does not include a requirement to re-approve 
these programs. Furthermore, the Ministry 
does not know how old the majority of these 
older programs are, nor does it have a formal 
plan to call in these programs for re-approval. 

• Although a recent risk assessment completed 
by the Ministry identified 180 private career 
college campuses with multiple compli- 
ance risk factors, the Ministry could not 
demonstrate that it had undertaken enough 
compliance inspections to adequately manage 
the risk of non-compliance with the Act and 
its regulations. For instance, although there 
are approximately 470 registered colleges 
and 650 campuses in Ontario, the Ministry 
estimated that only 30 campuses had been 
inspected in 2010, although we were informed 
that ministry investigators made 20 additional 
campus visits to address specific concerns. We 
also found that inspectors did not maintain 
sufficient documentation showing the full 
nature of the work performed. In addition, 
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Detailed Audit Observations 



according to the Ministry, just 5% of inspect- 
ors' time was devoted to inspections, as the 
majority of their time was spent on processing 
program approvals. 



ERALL MINISTRY RESPONSE 



The Ministry appreciates the recognition pro- 
vided by the Auditor General of the multiple 
initiatives that are under way to advance 
its oversight of private career colleges. The 
Ministry also agrees that student protection 
continues to be a fundamental priority. The 
Ministry is supportive of the Auditor General's 
recommendations and offers the following 
context. 

Supporting an expanded range of high- 
quality programs for students, while ensuring 
that illegal and non-compliant programs and 
operators are addressed, are dual priorities for 
the Ministry. Beginning with the creation of the 
new Private Career Colleges Branch in December 
of 2009, the Ministry has significantly strength- 
ened sector oversight and student protection, 
while working with the sector to improve the 
quality of programs and program applications. 

Through the use of the full spectrum of 
enforcement tools provided for in the legisla- 
tion, the Ministry has identified and taken 
enforcement action against over 150 illegal 
operators. The Ministry has also approved 944 
new programs in registered private career col- 
leges in the past 12 months, which will expand 
the options available for private career college 
students. The Ministry continues to evaluate 
existing policies, processes, and tools to ensure 
that they remain applicable and relevant to sec- 
tor oversight and student protection. 

In late 2011, the Ministry will begin a review 
of the Act and related regulations. This will 
provide an opportunity for the Ministry to 
reassess and validate the degree of oversight of 
the sector, given the ongoing need for student 
protection. 



UNREGISTERED PRIVATE TRAINING 
INSTITUTIONS 

Partly in response to recommendations made in a 
report issued by the Ontario Ombudsman in July 
2009, the Ministry has undertaken several initia- 
tives to address unregistered private training insti- 
tutions that offer unapproved vocational programs. 
These initiatives include: 

• establishing the Private Career Colleges 
Branch, which is dedicated to the oversight 
of the sector and the investigation of unregis- 
tered institutions; 

• developing a regulatory framework to allow 
the Ministry to issue administrative monetary 
penalties to institutions that violate the Act or 
its regulations; 

• working with professional and regulatory 
bodies to improve communication and to 
strengthen awareness of each other's require- 
ments; and 

• significantly increasing enforcement action 
against unregistered private training 
institutions. 

To illustrate increased enforcement action, in 
a little over a year and a half (between August 1, 
2009, and March 31, 2011), the Ministry issued 
about 130 orders to unregistered private training 
institutions, or more than four times as many as 
in the preceding three-year period. In addition, 
starting in December 2009, shortly after developing 
the regulation to do so, the Ministry began to 
issue administrative monetary penalties to non- 
compliant institutions. From December 1, 2009, 
through March 31, 2011, the Ministry issued about 
120 notices of contravention and associated admin- 
istrative monetary penalties to unregistered private 
training institutions for violating the Act and its 
regulations. 

Within the Ministry's Private Career Col- 
leges Branch, the Compliance and Enforcement 
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Unit — which is staffed with four investigators, a 
manager, and a research analyst — is responsible for 
investigating allegedly unregistered private training 
institutions, as well as investigating allegations of 
major non-compliance issues at registered private 
career colleges. The vast majority of the Unit's 
investigations involve unregistered private training 
institutions. According to the Ministry, most of the 
investigative efforts involving unregistered private 
training institutions were reactive, based primarily 
on responding to tips and complaints from regis- 
tered private career colleges. 

Although our review of the Ministry's investiga- 
tions of unregistered private training institutions 
identified these improvements, there are several 
areas that warrant management attention: 

• The Ministry had not kept a record of tips 
and complaints containing allegations about 
unregistered private training institutions, 
although during the course of our audit the 
Ministry began to keep a centralized record of 
such tips and complaints. 

• The Ministry had not set a targeted time frame 
for completing investigations of allegations 
against unregistered institutions, and the 
Ministry had not tracked the time taken to 
complete such investigations so that it could 
establish a baseline against which a target 
could be set and subsequent performance 
measured. In about half the cases reviewed, 
there was insufficient information available to 
determine the length of time it took to com- 
plete investigations. Where an investigation's 
length could be determined, we found that on 
average, the Ministry took approximately 70 
days to complete the investigation and take 
enforcement action from the date the com- 
plaint had been received: investigation lengths 
ranged from fewer than 10 days to almost 220 
days. Ministry management informed us that 
in some of the lengthier cases, investigative 
and/or enforcement action had been delayed 
by the Ministry so that it could undertake 
simultaneous sector-specific investigations. 



However, the resulting delays could have put 
prospective students of these institutions at risk. 

• We reviewed a sample of investigations that 
had been closed because the Ministry had 
obtained documentation indicating compli- 
ance from unregistered institutions. However, 
we were told that subsequent follow-up to 
determine continued compliance was not 
strictly required: additional procedures and 
associated timelines were at the discretion of 
ministry investigators. In more than half of 
the investigations we examined, we observed 
that subsequent follow-up procedures to 
ensure continued compliance either had not 
been documented or had not been undertaken. 

• The Ministry was tracking information related 
to enforcement action against unregistered 
institutions and recording that information 

in a spreadsheet program. Management used 
this information to periodically analyze the 
impact of administrative monetary penalties, 
including measuring the number of such pen- 
alties issued per month and the impact of such 
penalties on encouraging compliance. This 
was a good initiative, but our review of the 
spreadsheets revealed errors and omissions 
that reduced the usefulness of the analysis 
undertaken. 
Management shared with us various proactive 
measures it would undertake if resources permit- 
ted, including advertising to prospective students, 
reviewing advertisements in ethnic newspapers to 
identify suspected unregistered institutions, and 
educating prospective students at events such as 
job fairs on how to differentiate registered and 
unregistered institutions. Although some proactive 
measures for identifying unregistered private train- 
ing institutions carry an additional financial cost, 
we observed that the Ministry had at its disposal 
information that could be used to identify possible 
unregistered institutions. Examples are information 
about programs pre-screened to determine whether 
they are vocational and require approval under the 
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Act, and information on institutions that ministry 
records indicate had closed. 

Subsequent to our discussions with ministry 
management, we observed that the Ministry began 
to review institutions that had pre-screened pro- 
grams during 2010 to determine whether they were 
offering and/or advertising unapproved vocational 
programs. As a result of this new process, the 
Ministry found a number of institutions that war- 
ranted further investigation and some cases that 
warranted enforcement action. 

At the outset of our audit, the Ministry had not 
undertaken a review of closed schools to ensure 
that they were not continuing to operate and adver- 
tise themselves as registered private career colleges. 
We reviewed a sample of the private career colleges 
that, according to ministry records, had closed since 
the beginning of the 2006/07 fiscal year and found 
examples where these institutions appeared to have 
continued to operate and advertise unapproved 
vocational programs. We informed the Ministry, 
and in two of these cases the Ministry subsequently 
took enforcement action against these institu- 
tions for infractions that included advertising and 
operating an unregistered private career college 
and advertising and/or providing unapproved voca- 
tional programs. During the course of our audit, 
the Ministry informed us that it had launched an 
initiative to review institutions that had closed, but 
at the completion of our fieldwork the results of this 
review were not available. 



RECOMMENDATION 1 



To enhance protection for current and pro- 
spective students of private career colleges, the 
Ministry of Training, Colleges and Universities 
(Ministry) should: 

• use the information at its disposal to 
proactively identify possible unregistered 
private training institutions offering or 
advertising unapproved vocational programs 
and establish a targeted time frame for com- 
pleting investigations; and 



• consider establishing standardized follow- 
up procedures and timelines to ensure that 
the unregistered institutions against which 
it has previously taken enforcement action 
continue to comply with the Ministry's 
requirements. 

| MINISTRY RESP0NSI 

The Ministry agrees with the Auditor General's 
recommendation and has implemented proto- 
cols that will allow it to fully utilize available 
information to enhance student protection. 

The Ministry consistently surveys available 
media, including the Internet, to find evidence 
of unregistered institutions. The Ministry also 
investigates unregistered institutions that have 
been identified by students or registered institu- 
tions. These sources of information have been 
the primary basis for the over 100 enforcement 
actions taken in the first nine months of 2011. 

In addition, the Ministry is tracking all: 

• unregistered institutions that have been 
subject to an enforcement order to ensure 
compliance and to detect new activity; 

• institutions that have applied for registration 
and been rejected or that have failed to com- 
plete their registration to ensure that they 
are not operating illegally; and 

• institutions that have surrendered their 
registration to ensure that they are not con- 
tinuing to operate illegally. 

The Ministry has reviewed all such circum- 
stances since 2006 and has initiated 30 investi- 
gations and six enforcement actions as a result. 

The Ministry is also expanding its scope by 
partnering with other law enforcement agencies 
for investigations, including investigations of 
immigration-related offences and fraud. 

Moving forward, the Ministry will build 
on its experience in investigations to develop 
benchmarks for investigation timelines. 
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PERFORMANCE MEASURES 

Although the Ministry collects performance infor- 
mation related to student outcomes for universities 
and public colleges, the Ministry does not currently 
collect performance information for private career 
colleges. For public colleges, the Ministry publishes 
information on five performance indicators: gradua- 
tion rates, graduate employment, graduate satisfac- 
tion, employer satisfaction, and student satisfaction. 
Such information helps students make informed 
choices regarding which colleges and programs will 
best meet their educational and employment goals. 
Similar measures would be helpful to the Ministry 
in overseeing private career colleges and meeting its 
primary objective of protecting students. 

Until 2006, the Ministry collected graduation 
and graduate employment rates for OSAP-eligible 
private career colleges. However, in 2006, what 
was to be a temporary moratorium was placed on 
the collection of these data to develop performance 
indicators for all private career colleges. The Min- 
istry subsequently developed performance indica- 
tors related to student outcomes for private career 
colleges similar to the measures in place for public 
colleges. However, more than four years after the 
Ministry placed a moratorium on the collection of 
student outcome data, the Ministry has yet to estab- 
lish a timetable for the implementation of these 
performance measures. 

Our survey of 500 recent private career college 
graduates funded by the Ministry's employment 
training programs found that over 85% of respond- 
ents felt that performance measures such as those 
developed, but not yet implemented, would be 
useful in deciding which private career college and 
program to select. About half of those we surveyed 
indicated that such information was currently 
being provided in some form by the colleges they 
attended. Our survey also found that, although 
75% of graduates were employed full- or part-time, 
less than 40% of graduates were employed full-time 
in a position related to their private career college 
program. This statistic clearly demonstrates that 



there is a need for consistent, comparable informa- 
tion that will enable prospective students to make 
informed decisions regarding which private career 
college and program to select. 

In addition to helping prospective students 
make informed choices, student outcome data can 
also be used by the Ministry to better ensure the 
effective use of public funds provided to students 
who attend private career colleges. Specifically, 
such information would assist the Ministry in its 
monitoring activities and enhance accountability 
for the expenditure of hundreds of millions of dol- 
lars of public funds provided to students attending 
private career colleges. For example, a private 
career college, to obtain initial OSAP approval, 
must demonstrate that its graduation and graduate 
employment rates for the previous two years are at 
least two-thirds of the average rate attained by pub- 
lic colleges. Such a requirement provides a measure 
of assurance to OSAP students that they have a rea- 
sonable likelihood of finding employment. It also 
enhances accountability for public funds. Although 
OSAP provides a combination of loans and grants 
to students to attend private career colleges, fund- 
ing provided to students through the Ministry's 
employment training programs was generally not 
repayable. These programs had no supplementary 
eligibility requirements that private career colleges 
must meet in order for the Ministry to provide stu- 
dents with funding to attend these institutions. 



RECOMMENDATION 2 



To help prospective students make informed 
decisions on which private career college and 
which program to enrol in, the Ministry of 
Training, Colleges and Universities (Ministry) 
should collect, validate, and publish student 
outcome data such as information on gradua- 
tion rates and employment in their field of 
study. In addition, the Ministry should use 
these data to assist in its oversight of the private 
career college sector. 
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The Ministry agrees with the Auditor General 
on the importance of collecting, validating, and 
publishing private career college performance 
outcomes. The Ministry has completed a review 
of the former key performance indicator process 
and has developed six performance indicators 
for the private career college sector. The Min- 
istry will be working with the sector associations 
on developing a phased implementation plan 
that will moderate the financial and administra- 
tive impacts for both the Ministry and private 
career colleges. 

In addition, the Ministry has now imple- 
mented performance measures such as the 
Student Satisfaction Survey for students 
affected by a school closure and protected under 
the Training Completion Assurance Fund. The 
survey measures student satisfaction with the 
Ministry's support, the student's program, and 
the quality of education received from the train- 
ing completion provider. 



REGISTRATION 

Registration Applications 

The Ministry's Private Career Colleges Branch is 
responsible for the regulatory tasks associated 
with administering the Private Career Colleges Act, 
2005. The Act generally requires that institutions 
providing instruction in the skills and knowledge 
necessary to get a job in a vocation must be regis- 
tered and that their vocational programs must be 
approved by the Ministry. As a first step toward 
registering as a private career college, institutions 
complete the Ministry's pre-screening process. 
Based on information provided through this pro- 
cess, the Ministry determines whether a program 
is vocational, and thus whether the institution is 
required to submit an application for registration 
and program approval. 



Applications for registration require the 
applicant to provide information and supporting 
documents, including an applicant profile, sample 
student contracts, copies of the student complaint 
procedure, financial information and security, 
and insurance coverage. We reviewed a sample of 
approved registration applications and found that 
the required documentation was generally on file, 
but we had several concerns about the adequacy of 
some of the documentation obtained and the pro- 
cedures undertaken by the Ministry to assess that 
documentation: 

• According to the Act, the Ministry must be 
satisfied that the applicant can be expected 
to be financially responsible in the operation 
of a private career college. To assess financial 
viability, the Ministry requires applicants to 
provide forecast financial statements prepared 
by a public accountant. Although we observed 
that financial statements were on file, we did 
not see any documented evidence that such 
information had been reviewed. Reviewing 
available financial information is important 
given the impact a college failure can have on 
students, as well as on the Training Comple- 
tion Assurance Fund. We also observed that 
although applicants are asked to disclose if 
they have previously filed for bankruptcy, or 
are currently party to bankruptcy proceed- 
ings, applicants' past credit history is not 
validated through procedures such as a credit 
check. 

• The Act requires that the Ministry be satisfied 
that applicants will operate the private career 
college "in accordance with the law and with 
integrity and honesty." To satisfy this require- 
ment, the Ministry asks applicants to submit 
references and disclose any criminal offences. 
Although a poor reference or a criminal record 
is not necessarily grounds to reject a registra- 
tion application, for the cases we examined 
the Ministry did not contact references to help 
establish the applicant's integrity and honesty. 
The Ministry also did not perform criminal 
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background checks to confirm applicants' 
assertions that they did not have a criminal 
record. 

• The Ministry did not maintain — and due to 
systems limitations was unable to generate — a 
reliable record of registration applications 
that had been rejected. Maintaining a list of 
rejected applications could enable manage- 
ment to perform follow-up procedures to 
ensure that these applicants and institutions 
do not subsequently operate and provide voca- 
tional programs in contravention of the Act. 

Registration Renewal 

Private career colleges are required to complete 
an annual application to renew their registration. 
Applications for renewal must be accompanied by 
supporting documentation, such as financial state- 
ments prepared by a public accountant, a schedule 
of domestic and international student enrolment, 
and a continuation certificate with respect to finan- 
cial security posted in the form of a surety bond or 
letter of credit. 

We reviewed the registration renewal process 
and observed that the Ministry was generally doing 
an adequate job of obtaining financial statements, 
the appropriate amount of financial security, and 
proof that a continuation certificate was on file. 
However, we were concerned that in a few cases 
the Ministry did not ensure that financial state- 
ments had been prepared by a public accountant 
and the Ministry did not have a process in place for 
reviewing the submitted financial statements to 
determine if financial viability was in question. 

Our review of financial statements identified no 
significant concerns regarding financial viability in 
the vast majority of cases examined. However, our 
testing and discussions with responsible staff con- 
firmed that they were not assessing financial viabil- 
ity before renewing the registration of a private 
career college. Renewing a private career college's 
registration when it may not be able to meet its 
financial obligations could have an adverse impact 



on prospective students and the Training Comple- 
tion Assurance Fund. For example,we observed 
a private career college that had its registration 
renewed after it had considerable losses in its first 
year of operation and appeared to be dependent on 
shareholder loans. However, there was no indica- 
tion that the Ministry had taken any steps to obtain 
assurance that this college could be expected to 
meets its financial obligations. According to the 
Ministry, this college subsequently closed due to 
declining enrolment, and the Training Completion 
Assurance Fund had to pay out over $800,000. The 
Ministry advised us that this college's inability to 
meet new regulatory requirements may also have 
contributed to its closure. 

Management acknowledged that the Ministry 
did not have a staff member qualified to undertake 
an assessment of financial viability, but advised us 
that an individual would be hired to perform this 
task. 

Another criterion for renewing the registra- 
tion of a private career college is that the Ministry 
must be satisfied that the applicant will operate 
the college in compliance with the Act and its 
regulations. We noted that, although compliance 
issues appeared to have been considered during the 
registration renewal process, on-site inspections of 
colleges were limited. Although no formal record 
of field inspections was maintained, the Ministry 
estimated that it may have visited 30, or about 5%, 
of the 650 college campuses in 2010. 

With regard to the timeliness of registration 
renewal, colleges that apply for renewal remain 
registered until the Ministry makes a decision on 
their applications. The Ministry does not report 
on the length of time it takes to review and make 
decisions on applications for registration renewal, 
but we found that the renewal process could take 
more than a year to complete and averaged about 
150 days. Since private career colleges remain 
registered while their renewal application is being 
reviewed, prospective students could be put at 
risk should the Ministry decide not to renew their 
registration. In one case, where a registration had 
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expired more than seven months previously, the 
college's ability to meet its obligations appeared to 
hinge on continued shareholder loans. In another 
instance involving a registration that had expired 
more than seven months previously ministry 
inspections had identified serious concerns about 
misconduct, as well as numerous straightforward 
compliance issues. However, although at the time 
of our audit this college was under investigation, 
the Ministry had taken no enforcement action. As a 
result, prospective students could enrol at this insti- 
tution with no knowledge of these incidents. 



RECOMMENDATION 3 



To safeguard government funding provided to 
students and the money in the Training Comple- 
tion Assurance Fund as well as to enhance the 
protection offered to prospective students of 
private career colleges, the Ministry of Training, 
Colleges and Universities should: 

• ensure that its review of applications for 
private career college registrations is initiated 
on a timely basis and includes an appropriate 
assessment of the applicant's forecast finan- 
cial information, and checks on the appli- 
cant's references, credit, and criminal record; 

• maintain a record of rejected applications to 
facilitate management follow-up to ensure 
that rejected institutions do not subsequently 
operate in contravention of the Private 
Career Colleges Act, 2005; and 

• ensure the timely review of applications for 
registration renewal, including an adequate 
assessment of financial and other application 
information. 



The Ministry agrees with the Auditor General's 
view on the importance of verifying institutions' 
financial viability in order to protect prospective 
and current students. The Ministry has hired an 
accounting professional to assess the financial 
viability of private career colleges through a 



review of audited financial statements. This will 
allow the Ministry to proactively identify high- 
risk institutions. The Ministry complemented 
this process with a comprehensive three-tier 
financial review methodology, which was imple- 
mented in September 2011, for registration 
renewal. A similar framework for new appli- 
cants will follow. 

The Ministry has also developed operational 
policies and benchmarks to assist with meet- 
ing service timelines for the initial review of a 
registration application of a new private career 
college, prior to assigning it to an inspector. 
In 2011, the Ministry registered over 40 loca- 
tions. The Ministry's ability to meet its service 
commitments is directly related to the quality 
of the applications received. The Ministry will 
continue to work with private career colleges to 
improve the quality of submitted applications. 

As indicated in the response to Recom- 
mendation 1, the Ministry will track rejected 
applicants to ensure that they are not operating 
in contravention of the Act. 



PROGRAM APPROVAL 

Program Applications 

Applications for program approval require the appli- 
cant to provide information and supporting docu- 
mentation related to several aspects of the program, 
including admission requirements, program fees, an 
employment profile, and an outline of each subject. 
Legislation also requires applications to include a 
program evaluation report, submitted directly to the 
Ministry by the evaluator, who must have expertise 
in evaluating such programs. Evaluators assess the 
program's adequacy and recommend whether or 
not the program should be approved. For a program 
to be approved under the Act, the Ministry must be 
satisfied that the program will provide the skills and 
knowledge for students to obtain employment in the 
prescribed vocation. During the course of our audit, 
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the Ministry approved approximately 60 programs 
per month. 

We observed that the degree of assurance 
obtained by the Ministry before approving a 
program varies. For instance, some programs are 
evaluated by the regulatory body that oversees 
the profession or by a party recommended by the 
regulatory body. In the absence of a regulatory 
body, other programs are evaluated by experts in 
the subject matter against a defined ministry pro- 
gram standard or another formal standard that the 
Ministry recognizes or requires programs to meet. 
A third process is for programs to be evaluated by 
program assessors who have been pre-approved by 
the Ministry. However, according to the Ministry, 
in the vast majority of cases, the Ministry relies on 
a program evaluation completed by a third-party 
program assessor chosen by the college. 

Although the Ministry prescribes requirements 
that general third-party program assessors must 
meet, including adult education experience and 
professional experience in the field, we had the 
following concerns about programs reviewed by 
general third-party assessors. 

• Assessors are required to submit a resume 
and/or a summary of qualifications, but in the 
cases we reviewed there was no documented 
evidence that the Ministry had confirmed the 
credentials of the general third-party program 
assessors. Furthermore, staff responsible 

for reviewing program evaluation reports 
and assessor qualifications confirmed that 
these assessors' credentials were not usually 
checked. However, the Ministry informed us 
that they had begun to keep track of those 
assessors whose credentials had been valid- 
ated so this information would be available 
for subsequent evaluations. 

• To ensure that program evaluations can be 
relied on, the Ministry requires that all evalua- 
tions be undertaken by an arm's-length third 
party. The program application specifically 
notes that assessors should not have had any 
connection to the private career college being 



evaluated or to the program being reviewed 
within the last seven years. Both the applicant 
and the program assessor are required to 
declare that they are not in such a conflict-of- 
interest situation. However, for almost every 
application we reviewed, neither the applicant 
nor the assessor had declared that they were 
not in a potential conflict-of-interest situation. 
Although conflicts can be difficult to identify 
in the absence of a declaration, we noted an 
instance where the program assessor had 
been employed by the private career college 
in question within the previous two years and 
had been involved in developing curriculum 
at the college. There was no evidence on 
file that ministry staff had questioned this 
circumstance. 
We noted that the Ministry's information system 
did not include a record of whether approved 
programs had ever undergone a formal program 
evaluation. Although the Ministry informed us that 
to its knowledge, program evaluations had been 
required for many years, the Act and regulations 
preceding the current Act did not strictly require a 
formal program evaluation. We attempted to deter- 
mine whether older programs had been evaluated 
by a program assessor, but program information 
older than 10 years had been destroyed. 

For programs approved within the last 10 years 
but before the proclamation of the new Act, the 
Ministry was unable to provide documentation that 
the program had been evaluated in the majority 
of the cases we requested. However, where such 
documentation was provided, we observed that 
these programs had also been assessed by both 
employers and program design specialists. Employ- 
ers were asked questions such as whether the 
program covered the knowledge and skills required 
for entry-level employment. The Ministry used 
these assessments in deciding whether to approve a 
program. We felt this was a good practice, and our 
review of the practices of other Canadian jurisdic- 
tions revealed that some other provinces, unlike 
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Ontario, require employer program assessments to 
assist in the decision to approve a program. 

With regard to program applications, we were 
also concerned that the Ministry did not maintain a 
record of rejected program applications and could 
not generate a reliable record of applications that 
had been rejected. Maintaining a list of rejected 
program applications and the reasons for their 
rejection could enable management to perform 
follow-up procedures to ensure that such programs 
were not subsequently offered despite having failed 
to be approved by the Ministry. 



its review. These issues included missing documen- 
tation (such as program evaluations), unresolved 
compliance issues, and programs that did not 
meet current program standards. Despite these 
significant ministry improvements in reducing the 
backlog in processing program applications, our 
discussions with associations representing private 
career colleges indicated that they had concerns 
about the program approval process, including that 
the approval process was still too long and there- 
fore hindered the colleges' ability to respond to 
changing employment demands on a timely basis. 



Timeliness of Program Approvals 



Program Re-approvals 



We also reviewed the timeliness of the program 
approval process. We found that in response to 
a July 2009 recommendation from the Ontario 
Ombudsman to address delays in the time it takes 
to review and approve program applications, the 
Ministry had made progress in reducing the appli- 
cation turnaround time. In this regard, the Ministry 
also established a goal that no program application 
would await review by one of the Ministry's eight 
inspectors for longer than six months. In an effort 
to meet this goal, approximately 80% of inspector 
time was devoted to reviewing program applica- 
tions in 2010. Inspectors therefore spent very little 
time on their other responsibilities, such as visiting 
colleges to perform inspections, resolving student 
complaints, and assisting with college registration 
renewals. 

In August 2010, the Ministry began to track and 
report on a monthly basis the number and age of 
outstanding program applications. The Ministry 
noted that the percentage of program applica- 
tions awaiting review for longer than six months 
declined from 46% in August 2010 to an average of 
31% in the first quarter of 2011. Also, the number 
of applications older than one year declined from 
28% to an average of 13%. In addition, the Ministry 
advised us that a significant number of the applica- 
tions awaiting review for longer than one year had 
issues that prevented the Ministry from completing 



Most programs, to continue to provide the skills and 
knowledge necessary to obtain employment, will 
need to adapt over time to meet market demands. 
According to a regulation under the Act, which 
came into force on September 18, 2006, programs 
approved from that point on could be approved for 
a maximum of five years. Programs approved before 
the regulation came into force have no expiry date. 

As of March 31, 2011, there were approximately 
5,000 approved private career college programs. 
According to the Ministry, almost 2,000, or 40%, 
of the currently approved programs had been 
approved before the proclamation of the current 
Act and thus do not have an expiry date. The Min- 
istry was unable to provide us with a list indicat- 
ing the age of these older programs, because the 
approval dates for the vast majority of the older 
programs were not recorded in the information sys- 
tem. Our review of a sample of programs approved 
before the regulation came into force identified 
several programs that had been approved more 
than 20 years ago, including one program that had 
been approved almost 35 years ago. Although dis- 
cussions with the Ministry revealed that it intends 
to eventually call in such programs for re-approval, 
the Ministry did not have a documented plan or 
timetable for doing so. 

We reviewed the Ministry's full list of currently 
approved programs and found that, for programs 
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identified by the Ministry as having been approved 
after September 18, 2006, almost 30% did not have 
an expiry date recorded in the Ministry's informa- 
tion system. We also found that a number of pro- 
grams had an obviously incorrect program approval 
or expiry date, such as 1900 or 2099. In addition, 
in a number of cases the Ministry was unable to 
provide source documentation to substantiate the 
expiry date. Furthermore, in more than half the 
cases we examined where the program's expiry date 
was recorded in the system and the source docu- 
mentation was available, the expiry date recorded 
in the system did not match the source documenta- 
tion. Such data integrity issues affect the Ministry's 
ability to effectively manage re-approvals. 

Concerns over data integrity notwithstanding, 
we noted that over 80 programs were already flag- 
ged in the Ministry's information system as having 
expired during 2010 and about 90 were also set 
to expire in 2011. However, the Ministry did not 
have a documented plan for when it would call in 
these programs for re-approval. According to the 
Ministry, these programs will not expire and will 
remain approved until it decides to call them in for 
re-approval. 



RECOMMENDATION 4 



To enhance the quality of private career col- 
lege programs and to ensure that all programs, 
regardless of which college is offering them, 
provide the skills and knowledge currently 
necessary to obtain employment in the pre- 
scribed vocation, the Ministry of Training, Col- 
leges and Universities should: 

• review the processes in place to assess the 
qualifications and independence of the gen- 
eral third-party program assessors that pro- 
vide recommendations for program approval; 

• maintain a record of rejected program 
applications and consider implementing 
follow-up procedures to ensure that such 
programs are not offered despite their not 
being approved; 



• build on the progress made to date in 
improving the timeliness of the program 
approval process and develop a plan for pro- 
gram re-approvals; and 

• enhance its system so that it can provide the 
information needed to effectively manage 
the program approval process. 

| MINISTRY RESPONSE 

The Ministry agrees with the Auditor General's 
recommendation and, in September 2010, the 
Ministry started work on improving the third- 
party evaluation process, including a redesign 
of the Program Evaluation Report Form. The 
form will now include more substantial qualita- 
tive analysis as well as sections for commentary 
for both Adult Education and Subject Matter 
experts. The Ministry has also standardized the 
process for validating assessors' credentials with 
regulatory institutions and now tracks these 
assessors for future reference and evaluation. 

As indicated in the response to Recommen- 
dation I, the Ministry has also begun to track 
rejected program applications to ensure that 
these programs are not being offered in contra- 
vention of the Private Career Colleges Act, 2005. 

In the last 12 months, the Ministry has 
approved a total of 944 program applications. 
The Ministry continues to work toward a service 
delivery standard of six months for decisions on 
programs in queue. In the last year, the Ministry 
has met this standard for 94% of complete 
applications. 

While the Ministry has made significant 
progress in processing program approval appli- 
cations, the ability of the Ministry to improve 
turnaround times for program approval is dir- 
ectly related to the quality of the information it 
receives from the sector. The Ministry will con- 
tinue to work closely with the sector to improve 
the quality and completeness of information 
received. 
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While under the current legislation the 
Superintendent is not required to re-approve 
pre-2006 programs, the Ministry appreciates the 
Auditor General's concern for the quality of older 
programs and will explore options for program 
re-approval as part of its operational planning 
and in preparation for the upcoming Act review. 

Finally the Ministry is actively working on 
developing options for a new information system 
that will facilitate program approval, improve 
feedback to institutions on the elements required 
for a complete program application, and 
enhance reporting and information retrieval. 



LEGISLATIVE COMPLIANCE 

Compliance Inspections 

The Ministry's Private Career Colleges Branch is 
responsible for inspecting private career colleges. 
Inspections focus on a number of areas, includ- 
ing student contracts, admission requirements, 
instructor qualifications, advertising, the procedure 
for handling student complaints, program compli- 
ance, and insurance requirements. Such inspections 
enhance the Ministry's ability to protect current 
and prospective students of private career colleges. 
The responsibilities of the Branch's eight inspectors 
who are charged with undertaking such inspections 
also include reviewing registration, renewal, and 
program applications, as well as resolving student 
complaints. 

We observed that at the outset of our audit, 
the Ministry did not have any goals in place with 
respect to the number of inspections (such as to 
inspect all private career colleges over a defined 
period of time or to inspect colleges identified as 
high risk) . We requested a list of inspections that 
had been completed during the previous three fiscal 
years and up to the end of the calendar year 2010. 
However, the Ministry was unable to provide us 
with such a list. In the absence of a list of inspec- 
tions, management estimated that during 2010, 



about 30 campuses had been inspected. We were 
informed that ministry investigators had made 
20 additional campus visits to address specific 
concerns. 

The Ministry indicated that resource constraints 
were the reason so few inspections had been per- 
formed in the previous year. According to manage- 
ment, 80% of inspectors' time had been devoted to 
reducing the backlog of program applications. As a 
result, management estimated that inspectors spent 
just 5% of their time on inspections, even though 
management noted that compliance inspections 
were an important monitoring function to protect 
current and prospective students. To be effective, 
management told us, between 150 and 200 inspec- 
tions should be conducted annually. Furthermore, 
management was of the opinion that each regis- 
tered private career college should be inspected at 
least once every three years. 

We noted that the Ministry had prepared a risk 
assessment of Ontario's private career colleges in 
2009 that included identifying whether each col- 
lege posed a risk in eight defined risk categories, 
including chronic violation of the Act and its regula- 
tions, registration renewal issues, and questionable 
advertising practices. The Ministry's risk assess- 
ment identified 118 private career colleges that 
were chronic violators; as well, 77 had advertising 
issues, and 48 had registration renewal issues. We 
were informed that, since the Ministry's focus over 
the last two years had been on program approv- 
als, the risk framework was not used to schedule 
inspections. In addition, we were told that those 
inspections that did occur were often in response to 
specific concerns that arose during this time period. 
Nevertheless, we used the risk assessment to select 
a sample of private career college campuses with 
multiple risk factors and private career college 
campuses identified as chronic violators to deter- 
mine whether these schools had been inspected. 
We found that only one-third of the 60 colleges we 
selected had been inspected during the previous 
three calendar years. 
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During the course of our fieldwork, the Ministry 
completed an updated risk assessment that identi- 
fied about 470 private career college campuses 
with one or more risk factors, 180 with two or more 
risk factors, and 50 with three or more risk factors. 
For the 2011/12 fiscal year, the Ministry has com- 
mitted to tracking the percentage of higher-risk 
private career colleges that are inspected, and at 
the time of our fieldwork it indicated that it intends 
to inspect all private career colleges with three or 
more risk factors. 

Inspection Procedures 

Although we had concerns over the low number 
of inspections completed, we also had concerns 
about the quality and consistency of inspections in 
ensuring that private career colleges comply with 
the Act and its regulations. While the Ministry had a 
standardized checklist in place to guide inspectors, 
our review of the checklist and of recent inspections 
completed identified some inspection areas (such 
as advertising and instructor qualifications) where 
greater clarity or more detailed procedures would 
enhance inspection quality and consistency. For 
example, although instructor qualification require- 
ments are articulated in a regulation under the Act, 
the Ministry's checklist does not provide instruc- 
tions to inspectors on when and how to validate 
instructor qualifications. This observation was of 
concern given that among the private career college 
graduates we surveyed, those who did not feel that 
their program had provided good value relative 
to the fees they had paid cited the poor quality of 
instructors as the top reason for their dissatisfaction. 

With regard to advertising, the Ministry's 
checklist requires a review of the private career 
college's advertising materials. However, not all of 
the steps shown on the checklist are described in 
enough detail to communicate how to assess and 
reach a conclusion on advertising materials. To 
illustrate, one such question addresses the presence 
of a false or misleading statement, but generally the 
advertising materials reviewed by inspectors were 



not kept on file, so neither we nor ministry manage- 
ment are able to assess the adequacy of the testing 
performed. 

We were also concerned that inspections did 
not cover some significant areas outlined in the 
Act and its regulations. For example, no specific 
testing was done on the degree to which a program 
curriculum was being delivered as approved except 
with regard to the duration of the program — for 
example, to ensure that a full-time program was 
not being delivered on a part-time basis. In addi- 
tion, although inspectors ensure that the student 
complaint process meets legislative requirements, 
the Ministry's checklist does not require any specific 
testing to determine if actual complaints were being 
addressed appropriately by the college. Further- 
more, management indicated that inspectors 
are not generally expected to review curriculum 
except where a problem is suspected. In addition, 
management also noted that it does not have the 
requisite number of staff or the knowledge neces- 
sary to make such an assessment. We also noted 
that should the Ministry decide to undertake such 
assessments, in some cases its ability to do so would 
be compromised by the fact that it has destroyed 
program information older than 10 years. The lack 
of testing in these areas was a concern because the 
graduates we surveyed who felt that the program 
they had attended was not a good value cited the 
poor quality of curriculum as the second most 
prevalent source of their dissatisfaction. 

Management Oversight 

We noted a lack of management oversight of inspec- 
tions to ensure quality and consistency between 
inspectors. In addition, since details of inspection 
testing were not documented in some areas, man- 
agement's ability to monitor inspection quality was 
limited. Furthermore, management apprised us 
of some information system constraints that also 
limit its ability to undertake post-inspection quality 
control. Finally, due to the incomplete nature of 
its records, management had not aggregated the 
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results of inspections to identify trends and possible 
systemic issues that might warrant further investi- 
gation or amendments to inspection procedures. 

Our review of inspections and inspection reports 
identified several compliance issues, but the Min- 
istry did not deem most of these issues significant 
enough to warrant enforcement action. However, 
we did come across a situation where an inspector 
had discovered an unapproved program. This 
inspector identified the problem in the inspection 
report issued to the private career college, but the 
college continued to advertise the program and 
did not submit an application to have that program 
approved. Although the inspector subsequently 
requested that the college cease advertising the 
program and apply for program approval, the col- 
lege did not comply. However, the inspector did not 
bring this issue to the attention of ministry manage- 
ment. Consequently, this private career college con- 
tinued to offer this unapproved program for over 
a year. Only after we brought the situation to the 
attention of ministry management did the Ministry 
take enforcement action and issue a compliance 
order requiring the private career college to cease 
advertising the unapproved program. 

Although this was an isolated case among the 
files we reviewed, the circumstances that permit- 
ted its occurrence were systemic. We noted that 
inspectors are not required to bring inspection 
results to the attention of management. In addition, 
where compliance issues are found, the Ministry 
has not established timelines within which private 
career colleges must comply. 



RECOMMENDATION 5 



To enhance the level of compliance with the 
Private Career Colleges Act, 2005 and its regula- 
tions, and to provide better protection to stu- 
dents and prospective students of private career 
colleges, the Ministry of Training, Colleges and 
Universities should: 

• undertake enough inspections to adequately 
manage the risk of non-compliance; 



• clarify the focus and extent of testing that 
inspectors should perform during the course 
of an inspection of a college; 

• implement appropriate management over- 
sight procedures to enhance the quality and 
consistency of college inspections; and 

• aggregate and analyze inspection results to 
identify trends and systemic issues that war- 
rant further attention. 

| MINISTRY RESPONSE 

The Ministry agrees with the Auditor General's 
recommendation. 

In 2011, the Ministry assessed all registered 
private career colleges against a nine-part risk 
management framework for inspections that 
identified high-, medium-, and low-risk schools. 
The Ministry conducted inspections of all the 
private career colleges that were deemed "high 
risk." Management has established a goal of 
ensuring that all high-risk schools are inspected 
within three months of being identified and that 
all medium-risk schools are inspected within 24 
months of being identified. 

The Ministry has implemented additional 
financial risk review requirements to support 
the registration renewal process. Management 
will standardize reporting and oversight of 
the inspection process. Regular management 
reviews of inspection results, including trends 
in compliance issues identified through inspec- 
tions, have been introduced. The Ministry will 
strengthen the checklist and will expand the 
inspection protocol. 



STUDENT COMPLAINTS 

The Act requires every private career college to 
have a procedure in place for resolving student 
complaints. In addition, a regulation under the 
Act states that if a student is not satisfied with the 
way a private career college has resolved his or her 
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complaint, the student may refer the matter to the 
Ministry. We noted that, although the Ministry rec- 
ords written complaints from students in its infor- 
mation system, it does not keep a separate record 
of complaints that can be used to easily identify the 
number and types of complaints received. We were 
also advised that complaints from sources other 
than students are not recorded in the Ministry's 
information system. 

We also observed that the Ministry strives to 
acknowledge receipt of complaints within 15 busi- 
ness days, and it informed us that, beginning in the 
2011/12 fiscal year, it would track the percentage 
that had been acknowledged within five business 
days. Although the Ministry had not established a 
targeted time frame for resolving complaints, it did 
require that colleges establish a maximum length of 
time for making a decision on a complaint. In addi- 
tion, we noted that the Ministry had not set defined 
timelines for private career colleges to respond to 
requests for supporting documentation required 
by the Ministry, and the Ministry was not measur- 
ing the length of time taken to resolve complaints. 
Furthermore, we observed that management was 
not analyzing complaints to identify systemic issues 
or trends that require action on an overall basis. 

Our analysis of a sample of student complaints 
to the Ministry identified that resolving a student's 
complaint took from 17 to 244 days, or about 100 
days on average. Reasons for lengthy delays varied: 
for instance, in some situations the Ministry had 
difficulty obtaining the necessary information from 
the private career college, and in others the Min- 
istry could not demonstrate that it had reviewed 
information received from the private career col- 
lege on a timely basis. 

The Ministry has informed students of their 
ability to escalate unresolved complaints to the 
Ministry through means such as the Ministry's web- 
site and individual college complaint procedures. 
In addition, the Act requires that every student 
contract include an acknowledgement that the 
student has received a copy of the college's student 
complaint procedure and a copy of the "Statement 



of Students' Rights and Responsibilities" developed 
by the Ministry. Nevertheless, we were concerned 
that many students may not be aware that they can 
escalate their complaints to the Ministry. In the 
2008 through 2010 calendar years, the Ministry had 
received an average of about 80 student complaints 
annually, a number that is significantly less than 1% 
of students in private career colleges. Although this 
low rate might suggest that private career colleges 
are resolving student complaints in a satisfactory 
manner, our review of complaints and inspections 
identified cases where students seemed unaware 
that they could complain to the Ministry, because 
they initially directed their complaints to other 
sources (such as the Ontario Association of Career 
Colleges, the Better Business Bureau, and the Min- 
istry of Consumer Services) . In addition, our survey 
of private career college graduates revealed that 
of those students who filed a complaint with their 
college, 64% were not satisfied with the resolution 
of their complaint, yet only 14% of those individuals 
were aware that they could have escalated their 
complaint to the Ministry. 



RECOMMENDATION 6 



To help ensure that the protections offered by 
legislation to students of private career colleges 
are effective and to enhance management's 
ability to oversee the complaints process, the 
Ministry of Training, Colleges and Universities 
(Ministry) should: 

• establish target time frames for resolving 
complaints and for receipt from colleges of the 
information necessary to address complaints; 

• analyze complaints to identify possible issues 
or trends that may require more focused 
action; and 

• more effectively communicate to students 
that they are entitled to escalate unresolved 
complaints to the Ministry. 
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The Ministry agrees with the Auditor General's 
recommendation. 

The Ministry notes that private career col- 
leges are required, as a condition of registration, 
to include in every student contract a copy of 
the "Statement of Students' Rights and Respon- 
sibilities" developed by the Ministry and a copy 
of the private career college's student complaint 
procedure, expulsion policy, and refund policy. 
A private career college's student complaint 
procedure, expulsion policy, and refund policy 
must be aligned to the provisions within the 
legislation and approved by the Ministry. 

The Ministry has begun work on restructur- 
ing its public website to simplify access to infor- 
mation about how students may file complaints. 
The website will outline the student complaint 
procedure step by step, including how to submit 
complaints to the Ministry where required. The 
Ministry will also continue to partner with the 
sector and with other government consumer 
protection institutions to ensure that students 
are provided with consistent and accurate infor- 
mation on protections under the Private Career 
Colleges Act, 2005. 

The Ministry will establish target time 
frames for an initial response to student 
complaints and for private career colleges 
to respond to documentation requests. The 
Ministry will also review complaints to identify 
trends for further analysis or action. 



PUBLIC AWARENESS 

The Ministry has undertaken a number of initia- 
tives to enhance public awareness with the aim of 
financially and academically protecting students 
and prospective students of private career colleges. 
These efforts include: 

• posting "buyer beware" messaging on the 
Ministry's website as well as on Facebook that 



includes instructions to prospective students on 
how to ensure that the private career college 
program they select is approved under the Act; 

• setting up a Web search function that allows 
prospective students to verify that a private 
career college is registered and that its pro- 
grams are approved under the Act; 

• posting ministry enforcement orders and 
financial penalties issued to registered private 
career colleges as well as to unregistered 
private training institutions both on the Min- 
istry's website and on Facebook; and 

• distributing "buyer beware" posters and 
pamphlets to organizations such as high 
schools, immigrant settlement agencies, and 
employment resource centres that instruct 
prospective students on how to ensure that 
they select a registered private career college 
and an approved program. 

However, we noted that the Ministry has not 
undertaken an overall evaluation of its communica- 
tion efforts to determine the degree to which it is 
reaching students and prospective students. In addi- 
tion, the Ministry confirmed that it had not used 
information such as user feedback collected from 
its Web search function to assess the adequacy of its 
communication efforts. We noted that results from 
the first quarter of 2011 showed that over 30% of 
respondents did not feel it was easy to find the infor- 
mation they were looking for and did not find all the 
information they needed on the Ministry's website. 

As noted, the Ministry publishes a list of institu- 
tions against which it has taken enforcement action 
and issued financial penalties on its website and on 
Facebook. However, when we spoke to associations 
representing private career colleges, they were 
concerned that the Ministry does not differenti- 
ate between enforcement action taken against 
registered private career colleges and enforcement 
action taken against unregistered private training 
institutions. These associations felt that because 
the majority of enforcement action has been taken 
against unregistered private institutions, this 
approach has resulted in associating the legitimate 
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private career college sector with illegal operators 
Concern was also raised that the primary focus of 
the Facebook page on private career colleges was 
not a positive approach to educating prospective 
students about the sector. Rather, the site was 
perceived as being negative and served primarily 
to identify institutions against which enforcement 
action had been taken. 



RECOMMENDATION 7 



To enhance protection offered to students and 
prospective students, and to ensure that the 
private career college sector is not unfairly 
affected, the Ministry of Training, Colleges and 
Universities should: 

• periodically evaluate the effectiveness of its 
communication strategy to identify oppor- 
tunities for improvement in helping students 
choose the private career college and pro- 
grams that best meet their vocational goals; 
and 

• work with private career colleges and their 
associations to ensure that student-oriented 
communications are user friendly and com- 
municate in a fair and transparent manner 



the protections offered to students who 
attend registered colleges and programs. 

| MINISTRY RESPONSE 

The Ministry agrees with the Auditor General's 
recommendation and has partnered with sector 
associations to work on the information relat- 
ing to private career colleges on the ministry 
website. 

As required by the Private Career Colleges Act, 
2005, every private career college student con- 
tract includes a copy of the "Statement of Stu- 
dents' Rights and Responsibilities" developed 
by the Ministry and a copy of the private career 
college student complaint procedure, expulsion 
policy, and refund policy. These are important 
protections for students, and the Ministry will 
continue to reinforce the requirement to include 
them in the contract and will promote the avail- 
ability of these policies to private career college 
students. Moving forward, the Ministry will 
regularly review its communications strategy 
to inform the sector and students of upcoming 
initiatives and new or changing regulatory 
requirements. 



Chapter 3 



Ministry of Education 

Section 

■3.13 Student Success 

Initiatives 



Background 



The Ministry of Education (Ministry) administers 
publicly funded education in Ontario and is gener- 
ally responsible for developing the curriculum, 
setting requirements for student diplomas, and 
providing funding to school boards. Currently, 
Ontario has 72 publicly funded school boards, of 
which 70 have secondary schools, with more than 
700,000 students attending some 900 secondary 
schools. Since 2003, the Ministry has implemented 
a number of initiatives to help Ontario's secondary 
school students graduate with a high school dip- 
loma. Together these initiatives comprise Ontario's 
Student Success Strategy. The strategy's overall 
objective was to have 85% of high school students 
achieve a secondary school diploma by the end of 
the 2010/11 school year. 

A 2003 report commissioned by the Ministry 
titled Double Cohort Study concluded that at least 
25% of Ontario students who began grade 9 in 
1998/99 would leave school without a high school 
diploma. The graduation rate at the time stood at 
68%. The report further pointed out that if a stu- 
dent was falling behind by one credit in grades 9 or 
10, he or she was at risk of dropping out. Using that 
criterion, at the time, 27% of the students who had 
completed grade 9 and 40% of grade 10 students 



were at risk of not graduating because they lacked 
at least one course credit. 

These statistics prompted the government to 
establish the Student Success Strategy to improve 
student achievement and dramatically reduce the 
dropout rate. The strategy helps students tailor 
their education to individual strengths, goals, 
and interests, and encourages students who have 
left school to return and complete their diploma. 
To earn an Ontario Secondary School Diploma, 
students must successfully complete 18 compulsory 
and 12 optional courses, complete 40 hours of 
community involvement, and pass the grade 10 
provincial literacy test or course. 

The Ministry's Student Achievement Division 
holds the primary responsibility for developing, 
implementing, and monitoring Ontario's Student 
Success Strategy. Approximately 50 full-time 
division employees are involved in the delivery 
of the Student Success Strategy, including oper- 
ational and administrative staff. School boards and 
schools are responsible for the delivery of student 
success initiatives. Every board receives funding 
for one student success leader to help implement 
program initiatives in its schools and funding 
for one student success teacher per secondary 
school, who is responsible for providing supports 
to students at risk of not graduating. In addition to 
per student funding provided for student success 
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teachers, in the 2010/11 school year, the Ministry 
provided almost $130 million to school boards for 
the delivery of student success initiatives. 



Services Branch. We reviewed the branch's recent 
reports and considered its work and any relevant 
issues it identified in planning our work. 




The objective of our audit of the Ministry's Stu- 
dent Success Strategy was to assess whether the 
Ministry, selected school boards, and schools had 
adequate procedures in place to: 

• identify students at risk of not graduating and 
develop and implement initiatives to address 
their needs; 

• ensure that transfer payments are spent for 
the purposes intended and allocated based on 
student needs; and 

• measure and report on the strategy's effective- 
ness in increasing the number of students that 
graduate and are adequately prepared to pur- 
sue post-secondary education, apprenticeship, 
or employment. 

Senior management reviewed and agreed to our 
audit objective and associated audit criteria. 

Our audit work was conducted at the Ministry's 
head office, primarily in the Student Achievement 
Division, which is responsible for carrying out the 
Student Success Strategy, as well as at selected 
school boards and a sample of secondary schools in 
these boards. The boards we visited were the Lamb- 
ton Kent District School Board, the Simcoe County 
District School Board, and the Toronto Catholic 
District School Board. 

In conducting our audit work, we reviewed rel- 
evant legislation, policies, and procedures, and met 
with the appropriate ministry staff. We also met 
with school board staff, including principals and 
teachers. We researched related practices in other 
jurisdictions and solicited the opinions of faculty 
at universities and colleges about the level of pre- 
paredness of graduates from Ontario's secondary 
school system. Our audit also included a review of 
relevant activities of the Ministry's Internal Audit 




The Ministry set an overall objective whereby 85% 
of secondary students would graduate with a high 
school diploma by 2010/11. Based on the reported 
graduation rate, steady progress has been made 
toward achieving this goal: the graduation rate 
stood at 81% in the 2009/10 school year compared 
to 68% in 2003/04. Further refinements may be 
needed to the initiatives under way to ensure that 
the Ministry's objective can be met and that gradu- 
ating students have acquired the knowledge and 
skills needed for successful post-secondary study or 
employment. Some of our more significant observa- 
tions regarding the delivery of the Student Success 
Strategy were: 

• Overall, we found that the school boards 
we visited did a good job of identifying and 
providing supports to individual students 
considered at risk of not graduating. The 
boards and schools track risk factors such as 
gender, absenteeism, and course success to 
help identify students at risk. Although the 
boards we visited targeted most programming 
to individual students at risk, some other 
jurisdictions have found that more formally 
targeting supports to higher-risk groups of 
students based on such risk factors as ethni- 
city, disability, or economic status can be very 
effective in improving graduation rates. For 
example, targeted programming in one U.S. 
high school resulted in a 92% graduation rate 
for African-Americans far exceeding the state- 
wide average of 67% for this group. 

• The Ministry's reported graduation rate is 
based on calculating the percentage of grade 9 
students who graduate within five years. How- 
ever, the graduation rate would have been 
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72% had only four years of high school been 
considered, which could provide a measure 
for how well schools have delivered the cur- 
riculum. On the other hand, the graduation 
rate would have been 91% if the Ministry 
reported the overall graduation rate by the 
time students reach the age of 25, which 
would provide a better picture of the number 
of people in Ontario who have achieved at 
least a high school education. 
The Ministry and school boards are collecting 
useful information, such as credit accumula- 
tion rates, needed to identify students at risk 
of not graduating at an early stage and to 
track their progress as student success initia- 
tives are implemented and additional supports 
are provided. In the absence of any provincial 
testing beyond the grade 10 literacy test, the 
Ministry relies primarily on tracking changes 
in the graduation rate to measure the success 
of the strategy. However, unlike EQAO results, 
graduation rates are not publicly available 
by board, and school boards do not yet use a 
consistent method of calculating their gradua- 
tion rates to allow meaningful comparisons 
across the province. Better information is also 
needed on graduates' level of preparedness 
for post-secondary studies and employment. 
For the 2010/11 school year, a student re- 
engagement initiative encouraged more 
than 5,000 students to return to school to 
get their diploma, but in some cases the 40 
hours of community involvement was the 
only outstanding requirement to graduate. 
At one of the boards we visited, a school had 
implemented a program to help students 
complete their community involvement hours 
and, in the second year of the initiative, 
one-half of the grade 9 students had already 
completed this requirement. Furthermore, 
this school found that these students generally 
continued to participate in community activ- 
ities, accumulating hundreds of community 
involvement hours. 



• We noted situations where the Cooperative 
Education program documentation did not 
clearly demonstrate the link between the 
work placement and the associated curricular 
expectations as required. Cooperative educa- 
tion allows students to earn secondary school 
credits through a work placement related to a 
ministry-approved course. For example, stu- 
dents earned credits working in a wide range 
of placements, such as clothing stores, fast- 
food outlets, coffee shops, municipal planning 
offices, television studios, and laboratories. 
We found many examples where we ques- 
tioned whether the placement was directly 
related to the students' in-class curriculum 
learning expectations. 

• Over the past two school years, 2009/10 and 
2010/11, $15 million of the $245 million the 
Ministry provided to school boards for student 
success initiatives was allocated based on a 
direct assessment of student needs. Much 

of the funding was allocated based on the 
number of students enrolled in each board or 
based on applications submitted by boards. 
Although a considerable amount of this fund- 
ing is ultimately used to support students at 
risk, it was not necessarily targeted to the 
boards, schools, and students most in need of 
support. For example, a board with 81% of its 
students on track to graduate received $240 
per student, while a neighbouring board with 
only 69% of its students on track to graduate 
received less than half this amount. We were 
advised that the majority of the funding dis- 
parity was due to different degrees of board 
participation in programs funded through 
applications. 

• We found that the boards we visited properly 
accounted for Student Success funds received. 
However, in the last two school years ending 
in 2010/11, Ontario school boards received 

a total of nearly $8 million in unexpected 
funding late in the school year that had to be 
spent by year-end. Such late payments make 
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Detailed Audit Observations 



it difficult to effectively use this money, and 
some boards purchased items that schools did 
not necessarily need at the moment, such as 
more tools and more modern equipment. 



| OVERALL MINISTRY RESPONS 




We respect the recommendations of the Auditor 
General and have given careful considera- 
tion to their implementation. The Ministry of 
Education has three goals: to increase student 
achievement; to reduce gaps in achievement 
among students; and to increase public confi- 
dence in publicly funded education. Together 
with educators across the province, we place 
considerable importance on using research and 
outcome-based evidence to pursue these goals 
more effectively. The Auditor's report on the 
Student Success Strategy makes an important 
contribution to that effort. 

The report points to areas of commend- 
able practice as well as some specific areas for 
improvement. It points to school board practices 
that should be shared more widely. It supports 
the Ministry's and school boards' practices of 
collecting performance data and reporting 
on key indicators of progress. It supports our 
increased collaboration with the secondary 
education and post-secondary education and 
training sectors, and employers. It notes the 
progress made in increasing the graduation rate 
in Ontario and encourages our ongoing review 
of student success over a range of variables and 
time frames. Finally, it reinforces our commit- 
ment to policy and program assessment and 
affirms the importance of our ongoing program 
refinement to ensure that ministry and school 
board resources are collectively deployed to best 
meet student needs. 



MEASURING AND REPORTING ON 
STUDENT OUTCOMES 

Student Success Indicators 

The Student Success Strategy is a broad, province- 
wide strategy designed to ensure that students 
successfully complete their secondary schooling 
and reach their post-secondary goals. In 2005, the 
Ministry set a target of an 85% graduation rate by 
the 2010/11 school year. As noted in Figure I, the 
graduation rate has been steadily increasing, from 
68% in 2003/04 to 81% in 2009/10. 

In addition to monitoring the provincial gradua- 
tion rate, the Ministry collects information known 
as student success indicators to measure and 
evaluate student progress and to assess its Student 
Success Strategy. These indicators are based upon 
research conducted by the Ministry and other 
studies that highlight the factors that may eventu- 
ally result in students leaving school without a 
diploma. Some of the other indicators collected by 
the Ministry are credit accumulation rates, compul- 
sory course pass rates, and Education Quality and 
Accountability Office (EQAO) test results. 

The EQAO administers a province-wide grade 9 
mathematics test and the grade 10 Ontario Second- 
ary School Literacy Test (OSSLT). The EQAO reports 
publicly the results of these tests on a provincial, 
board, and school level. The grade 10 literacy test is 



Figure 1: Ontario High School Graduation Rates (%) 

Source of data: Ministry of Education 
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the last independent province-wide assessment of 
performance that secondary students are given. 

Although the Ministry publishes grade 10 credit 
accumulation rates by board, it does not publish 
graduation rates by board, nor has it established 
specific goals for any of its student success indica- 
tors. Establishing goals for individual indicators 
and measuring progress toward those goals at the 
school, board, and provincial levels can provide 
early warning signs that intervention may be 
required. 

Internally, the Ministry assembles board data 
reports on key indicators along with provincial 
averages. While the boards did not use these indica- 
tor data reports for analytical purposes, they did 
use them to verify and reconcile their own data and 
to compare them with provincial averages. How- 
ever, with the exception of shared grade 10 credit 
accumulation data, boards and schools cannot 
assess where they stand in relation to comparable 
boards and schools in other parts of the province. 

On a broader level, since the 2009/10 school 
year, boards have been expected to prepare overall 
improvement plans for their secondary schools, with 
clearly defined performance targets, intended strat- 
egies to achieve those targets, and relevant timelines 
for reporting on results. Ministry monitoring of these 
plans involves discussions with senior board staff 
three times a year regarding progress toward achiev- 
ing targets, lessons learned from the past year, and 
strategies to be implemented in the future. 

We reviewed a sample of school board improve- 
ment plans and noted that although some had 
common goals, the Ministry did not integrate them 
into an overall strategy. For example, most boards 
adopted the provincial graduation rate targets, but 
there was no attempt to outline how each board 
would contribute to achieving the overall goal of an 
85% graduation rate. Only one of the three boards 
we visited established specific and measurable tar- 
gets related to credit accumulation and graduation 
rates. 

We noted some other jurisdictions that had more 
rigorous accountability and transparency structures 



for their education ministries and individual school 
boards (authorities, districts) through formalized 
annual reports with long-term plans that included 
performance indicators and targets. Alberta, for 
example, uses a common and consistent set of 
performance measures where school authorities' 
performance measures are aligned with province- 
wide goals. Alberta also reports on additional per- 
formance measures such as annual dropout rates 
and post-secondary transition rates, while British 
Columbia requires its districts to outline how their 
strategies will be adjusted when targets have not 
been met. 

Graduation Rate Calculation and Reporting 

The Ministry's method for calculating the gradua- 
tion rate is based on a cohort approach that meas- 
ures the percentage of students who graduate with 
an Ontario Secondary School Diploma within five 
years after starting grade 9. There are some gaps 
inherent in the calculation such as not having the 
information to include students who may have 
graduated outside Ontario. Also, students may have 
left school and returned to finish their diploma in a 
year beyond the five years from when they started 
grade 9. 

Many factors need to be considered in deciding 
how best to calculate a graduation rate. Each meth- 
odology has advantages and disadvantages, and no 
method will produce a statistic that is ideal for all 
purposes. Although the high school curriculum has 
been designed so that it can be completed in four 
years, the Ministry has selected the five-year cohort 
rate as the official measure of student success. 
Reporting the four-year cohort rate would provide 
an assessment of how many students have com- 
pleted the curriculum — and how often schools have 
delivered the curriculum to students — within the 
four-year time frame. Also, another measurement 
such as reporting an overall high school graduation 
rate by a certain age — say, 25 — would provide a bet- 
ter picture of the number of people in Ontario who 



2011 Annual Report of the Office of the Auditor General of Ontario 



have achieved at least a high school level of educa- 
tion. (See Figure 2.) 

Two of the school boards we visited calculate 
multiple graduation rates such as four-year, five- 
year, and six-plus years to provide a broader picture 
of their performance. While the boards may calcu- 
late such rates, none of the ones we visited publish 
their graduation rates, although some of this infor- 
mation can be found in their board improvement 
plans. 

Furthermore, each of the school boards we 
reviewed utilized a different method for calculating 
its graduation rate. They ranged from variations 
of the Ministry's cohort method to not basing the 
calculation on a cohort at all. Therefore, even if 
graduation rate information were available, it 
would be misleading to use it to compare board or 
school graduation rates across the province. 

In addition to monitoring graduation rates, we 
noted a school board that gathered, at both the 
board and school level, other graduation-related 
statistics such as: 

• how many students attained their diplomas 
within four years; 

• how many students transferred elsewhere for 
educational purposes; 

• how many students did not return in the fall 
but received a diploma subsequently; and 

• the percentage of students who left the educa- 
tion system without graduating. 

This particular board felt that using different 
rates and breaking down those rates provided it 
with data useful in implementing the student suc- 
cess initiatives, as well as a more complete picture 
of student activity regarding graduation. The fact 
that school boards are taking it upon themselves to 
complete this type of evaluation shows its useful- 
ness and demonstrates the value of doing such 
analysis consistently and province-wide. 

In October 2009, the Ministry established a 
working group to review the calculation of board 
graduation rates. The group agreed upon a number 
of factors relevant to determining school board- 
level graduation rates, including standardized 



Figure 2: Differing Ontario High School Graduation 
Rates (2009/10 School Year) (%) 

Source of data: Ministry of Education and Statistics Canada 
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calculations that are consistent with those used to 
determine the provincial cohort graduation rate. 
However, the group did not consider the calculation 
and reporting of graduation rates for individual 
schools. Still, there was agreement that further 
detail should be provided concerning students who 
did not graduate, such as gender and immigration 
status, for planning and program development 
purposes. The last meeting of the working group 
took place in March 2010, although the Ministry 
informed us that a new committee was to be con- 
vened in September 2011. 

We noted other jurisdictions that publicly report 
detailed board-level and school-level graduation 
and dropout rates as part of their accountability 
requirements. For example, Saskatchewan reports a 
completion rate for each year ranging from students 
who graduate in three years or less to those who 
spent eight or more years to achieve their diploma. 
Similarly, Alberta reports three-, four-, and five-year 
completion rates. Furthermore, Alberta reports the 
number of students who dropped out or continued 
in school but did not earn a diploma. 

Student Success Data Collection 

In addition to Student Success Strategy funding, 
the Ministry has provided more than $120 million 
from the 2004/05 to the 2010/11 school year in 
funding to enhance the capacity of schools and 
school boards to use data and information for 
evidence-based decision-making to improve student 
achievement. In our audit, we noted that each of 
the three school boards we visited had a student 
information management system purchased from 
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a third-party vendor. These systems collected data 
and had the ability to produce board-, school-, and 
individual student-level reports, and to track and 
monitor student performance data for subsequent 
analysis. Although the Ministry has attempted to 
help build systems capacity at the board and school 
levels, a more province-wide approach might be 
more cost-effective given the similarities in the 
functionality of student information management 
systems. 

As noted earlier, each school board reports data 
to the Ministry on student achievement as well as 
attendance and biographical information, including 
country of birth and first language spoken at home. 
We found that the Ministry had a thorough process 
in place for ensuring that data collected by schools 
and school boards are verified so that the risk of 
inaccurate information is minimized. 

With the completion of Ontario School Infor- 
mation System (OnSIS) submissions by boards, 
indicator data can be finalized for internal decision- 
making and external reporting purposes. However, 
data are usually not finalized for more than six 
months after the submission date, because not all 
boards submit data on time. Consequently, the Min- 
istry provides boards with preliminary data to be 
used for analysis and program planning purposes. 

At each school board visited during our audit, 
we found that data had been verified prior to 
upload and sign-off for OnSIS. We also noted some 
practices aimed at enhancing the efficiency of this 
process, such as having built-in system verification 
that is run nightly to flag potential errors to initiate 
the ongoing correction of data. We also noted that 
one board provided release time for school staff 
to visit the board office to verify data and correct 
errors, an approach that assisted in the timeliness 
of data submission. All boards we visited conducted 
workshops and training sessions on data prepara- 
tion and verification. One of the school boards also 
conducted internal enrolment reviews to ensure the 
reliability of its data. 



Tracking Students after High School 

The Ministry has had to undertake surveys to assess 
the success of its initiatives in preparing students 
for apprenticeships, college, university, and the 
workforce because it is not possible to track stu- 
dents beyond high school, as information regarding 
graduates is not readily available through its cur- 
rent systems. 

The Ministry and the Ministry of Training, 
Colleges and Universities are working with col- 
leges and universities to facilitate the use of the 
Ontario Education Number as the common student 
number for students from kindergarten to college/ 
university graduation. This project has a target date 
for implementation in 2012. The current plan is to 
eventually include other government-sponsored 
employment training programs and apprentice- 
ships in the project. However, the project does not 
include extending the use of the Ontario Education 
Number to private career colleges. 

Information such as career choices, university/ 
college enrolment, post-secondary marks, and 
credentials and qualifications earned could help 
to evaluate the success of ministry initiatives and 
assess how well former students are performing 
after high school. Additionally, examination of stu- 
dent data related to students' performance in post- 
secondary pursuits, such as college, university, or 
apprenticeships, could help provide a better under- 
standing of the knowledge gaps and skills required 
to be successful in a post-secondary setting. 

British Columbia uses a common identifier 
number to track and report on graduating students' 
post-secondary destinations for up to seven years 
after high school graduation. California also 
uses a common identifier, and its post-secondary 
institutions report back to high schools on cur- 
riculum areas where students were not sufficiently 
prepared. Meanwhile, in Florida, an information 
system allows for the tracking of students from 
the time they first enter school until they enter the 
workforce. 
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RECOMMENDATION 1 



To help the Ministry of Education (Ministry), 
school boards, and schools generate timely data 
for decision-making purposes that are consistent 
and comparable, the Ministry and the province's 
school boards should: 

• set reasonable targets for graduation rates 
and student success indicators in line with 
overall provincial goals and require more 
formal reporting on the achievement of 
these targets at the provincial and school 
board levels; 

• develop a common method for school boards 
to calculate and report graduation rates and 
other student success indicators; 

• help school boards share best practices that 
would assist in the more timely verification 
and submission of student data; 

• consider collecting information on high 
school graduates to identify any gaps in 
knowledge or skills that may require atten- 
tion; and 

• extend the use of the proposed student 
identifier number to include private career 
colleges. 



| MINISTRY RESPONSE 




The Ministry agrees that consistent, comparable 
data are required for decision-making purposes 
and will work with school boards to identify 
and share effective data collection and verifica- 
tion practices. This work can then be used for 
decision-making processes such as establishing 
reasonable targets for board graduation rates, 
creating common calculation and reporting 
methods, and monitoring students as they 
progress into their chosen post-secondary path- 
ways. The Ministry will also continue to work 
with the Ministry of Training, Colleges and Uni- 
versities to extend the student identifier number 
to apprenticeships, colleges, government- 
sponsored employment training programs, and 



universities, and will explore the feasibility of 
extending it to private career colleges. 



STUDENTS AT RISK OF NOT GRADUATING 

Identifying Individual Students at Risk 

Students who are socio-economically disadvan- 
taged, as well as those who have behavioural traits 
such as high absenteeism and those from certain 
cultural backgrounds, are more likely than other 
students to experience difficulty in school and drop 
out. Studies also have shown that students at risk of 
not graduating benefit from early identification and 
intervention. 

For example, credit loss has been identified 
as one of the biggest factors affecting graduation 
rates. A 2009 study of Ontario students noted that 
one failed course in grade 9 reduces by more than 
20% an Ontario student's chance of graduation 
within five years. One U.S. city noted that only 
28% of its students who were off -track for required 
courses in grade 9 graduated from high school 
within five years. 

Overall, we found that the school boards we 
visited did a good job of identifying and provid- 
ing supports to individual students considered at 
risk of not graduating. In general, the boards had 
effective transition programs for students moving 
from grade 8 to their first year of high school in 
grade 9. In the three boards we visited during our 
audit, grade 8 teachers along with a high school 
student success teacher prepared detailed student 
profiles to be passed on to the student's grade 9 
teacher. The student profiles contain information 
regarding academic learning skills, at-risk behav- 
iours, strengths, needs, and suggested learning 
strategies. 

The boards also had processes to identify at-risk 
students and assigned each student to a high school 
staff member (Student Success teacher, guidance 
counsellor, regular teacher) to help them with any 
difficulties. In addition, two boards established 
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peer mentor programs to help guide these students 
through their first year of high school. 

Although each of the school boards we visited 
varied slightly in its approach, all boards identified 
at-risk students based on indicators such as credit 
accumulation, grades, attendance, suspensions, 
and EQAO results. In addition, one school board 
produced reports that broke down the number of 
students considered significantly at risk, moderately 
at risk, and on track. The school's Student Success 
Team would then work out strategies to assist the 
identified students at regularly scheduled meetings. 

School boards are required to report the number 
of at-risk students to the Ministry three times per 
school year so it can track these students on a prov- 
incial level. These reports also include what strat- 
egies are being used to keep the students in school, 
such as being assigned a high school staff member, 
having a strength-and-needs-based student profile, 
and establishing an education and career pathway. 
The Ministry does not formally compile these board 
reports, but it does prepare some provincial trend 
analyses. 

The Ministry has provided a common at-risk 
definition for grades 11 and 12 that boards are 
required to use. For grade 9, boards use their own 
definitions, and for grade 10, boards use credit loss 
and other locally determined factors to identify at- 
risk students. Note that Figure 3 shows a significant 
drop in the number of at-risk students in the later 
grades as compared to grades 9 and 10. However, 
it is not possible to determine if this drop is due to 
differing school definitions and methodologies or 
if early interventions are successful in reducing the 
number of at-risk students. 

Identifying At-risk Groups 

Academic research into Ontario's education system 
has identified some groups of high school students 
more at risk of not graduating than others. For 
example: 

• Male students are less likely to graduate than 
females. 



• Students from certain linguistic groups are 
less likely to graduate and go on to post- 
secondary education than others. 

• Rural and Northern Ontario students are 
less likely to apply to and register in post- 
secondary education than urban and southern 
Ontario students. 

Despite this evidence, information on gradua- 
tion rates is not differentiated by sub-categories. 
However, the Ministry does track certain gaps, such 
as by gender, through its student success indicators. 

One further step in identifying student groups 
at risk of not graduating involves extracting data 
based on factors such as ethnicity and language 
spoken at home. Although this is considered a 
sensitive issue, programs in other jurisdictions 
have found that supports targeted to specific ethnic 
groups can be constructive, because these types of 
data can help guide program planning and delivery. 
For example, targeted programming in one U.S. 
high school resulted in a 92% graduation rate for 
African-Americans, far exceeding the state-wide 
average of 67% for this group. 

One such initiative reported by the Toronto Dis- 
trict School Board in January 2011 studied achieve- 
ment test scores and completion-of-graduation 
requirements for self-identified students from Latin 
America who speak Spanish at home. The report 
noted that 40% of these students left high school 
before graduation. The report also noted that it 
was the first time that any Canadian school board 
had collected and extracted achievement data 

Figure 3: Percentages of Students at Risk of Not 
Graduating (2009/10 School Year) (%) 

Source of data: Ministry of Education 



Grade 




9 


23 


10 


21 


11 


13 


12 


17 


12+* 


18 



* Includes students still in secondary school and not having graduated 
after four years. 



2011 Annual Report of the Office of the Auditor General of Ontario 



based on students' self-identified ethno-linguistic 
background. As a consequence, the board reported 
that, among other initiatives, it would implement 
cultural sensitivity classes for teachers, offer sup- 
port programs for newcomers, and in some cases 
give students from lower-income households part- 
time jobs at their school. 

Currently, data collected by the Ministry in 
the Ontario School Information System (OnSIS), 
mainly through student registration forms, include 
language first spoken, residence status in Canada, 
year of entry into Canada, and country of birth. 
Although the Ministry informed us that it is not cur- 
rently possible to accurately calculate student suc- 
cess indicators by various groups, one of the school 
boards we visited did sort key student indicators by 
attributes such as country of origin and language 
spoken at home. 

Other jurisdictions have also managed to report 
indicators based on various student groups. British 
Columbia, for example, has reported that 76% of 
its students who speak East Asian languages gradu- 
ated and went on to post-secondary education. It 
has also broken this group into different East Asian 
national backgrounds/countries of origin and cal- 
culated graduation rates accordingly. 

Identifying Early School Leavers 

Tracking and analyzing why students leave school 
before they graduate helps boards to establish more 
timely and effective programs and supports to 
assist students at risk before they drop out. Schools 
record why students drop out of the education sys- 
tem through the use of a series of pre-established 
destination or exit codes. 

We analyzed the total number of recorded exits 
for the four school years beginning in 2006/07 on a 
board-by-board basis. Many students were coded as 
unknown because school boards and schools were 
unsure where the students went or for what reason. 
In addition, a large number of dropouts were coded 
as "other," a category used when there is no specific 
code for the reason that the student left school. We 



concluded that such coding lacks any useful mean- 
ing and cannot be used for any constructive analysis 
of why students drop out of school and what might 
be done to help keep such students in school. 

To assess the overall success of programs to 
help keep students in the education system, other 
jurisdictions such as Alberta calculate an annual 
dropout rate for students aged 14 to 18. However, 
the Ministry informed us that it does not calculate a 
dropout rate because of methodology concerns such 
as accounting for students who leave the province or 
who leave and return to school several times. 



RECOMMENDATION 2 



To help identify students and student groups 
at risk of not graduating who may benefit from 
additional and specific supports and programs, 
the Ministry of Education and the province's 
school boards should: 

• establish a common definition for reporting 
grade 9 and grade 10 students considered at 
risk of not graduating; 

• assess the viability of calculating student 
success indicators by a variety of attributes 
such as ethnicity, language, and socio- 
economic status, and consider a system or 
process for collecting data based on student 
self-identification; and 

• review the processes used to record students 
who leave school without a diploma so that 
the reasons students leave school can be 
determined. 

| MINISTRY RESPONSE 

The Ministry will work toward establishing a 
common reporting definition of students con- 
sidered at risk of not graduating to help improve 
its ability to identify such students. The Ministry 
will also explore the viability of collecting data 
on students who self-identify on a variety of 
attributes, and continue to review and initiate 
research regarding students who struggle to 
complete school or leave school early. 
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STUDENT SUCCESS STRATEGY 
INITIATIVES 

The Ministry has developed a number of initiatives 
to help keep students in school, re-engage students 
who have dropped out, and prepare students for 
post-secondary education, apprenticeship, and 
employment. For our audit, we reviewed six major 
ministry Student Success Strategy initiatives. 

• Re-engagement — Recent dropouts are encour- 
aged to return to school and complete their 
high school diploma requirements. 

• Cooperative Education — Students earn sec- 
ondary school credits for taking a job place- 
ment that enhances the classroom experience. 

• Credit Recovery — Students who failed a 
course are allowed to pass by working satisfac- 
torily on only those course expectations where 
the student had been unsuccessful. 

• Specialist High Skills Major (SHSM)— This 
career-focused program of courses allows 
students to earn related certifications while 
fulfilling graduation requirements. 

• Dual Credit — Students taking college or 
apprenticeship courses can earn credits for 
both their high school diploma and post- 
secondary qualifications. 

• Student Success School Support — A limited 
number of low-performing schools received 
funds for an additional Student Success leader, 
a mentor for the school principal, and addi- 
tional professional development opportunities. 

Re-engagement 

In August 2010, the Ministry implemented a re- 
engagement initiative to encourage recent dropouts 
to return to school and complete their high school 
diploma requirements. The Ministry calculated 
that 16,000 fourth-year students left high school in 
2010 without graduating but could have completed 
their diploma requirements with just another year 
of school. The Ministry provided each school board 
with a number of students whom boards were 



required to identify, contact, and attempt to bring 
back to high school. Boards were also expected to 
monitor these students' progress toward complet- 
ing their diploma and to place them in appropriate 
programs to maximize their chances of success. 

As of October 31, 2010, school boards had con- 
tacted more than 10,000 such students, and more 
than 5,000 had returned to school to complete their 
high school diploma. The boards we visited had 
put procedures in place to identify, contact, and 
monitor re-engaged students. Almost one-half of 
the returning students needed five or fewer course 
credits to graduate. 

In addition to course credits, 25% of the 
returning students needed to pass the grade 10 
literacy requirement, while more than 70% had not 
completed their 40 community involvement hours. 
The Ministry informed us that obtaining commun- 
ity involvement hours was the only graduation 
requirement that some of this 70% — it was unable 
to provide an exact number — needed to complete. 

One board said that many students had not com- 
pleted their community involvement hours because 
they lacked the resources or initiative to achieve 
this requirement on their own. In contrast, there 
is a significant school focus on passing the literacy 
requirement, with considerable support provided 
for students to pass this test. 

At one of the boards we visited, a school had 
implemented an initiative to help ensure that 
grade 12 students did not leave their community 
hours requirement to the last minute and jeopardize 
their graduation. The school encouraged grade 9 
students to complete their 40 hours in the first year 
of high school. The school presented students with 
various opportunities to obtain their hours through 
teacher-led activities and encouraged them to volun- 
teer in the community. Before this initiative began, 
fewer than 10% of grade 9 students had completed 
their community involvement hours. In the first year 
of the initiative, 25% completed their community 
involvement hours, and in its second year nearly 
half met the requirement. Furthermore, the school 
found that these students generally continued to 
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participate in community activities throughout high 
school, accumulating hundreds of hours. 

Overall, the boards told us that re-engagement 
was a worthwhile and successful initiative because 
it focused on a targeted group of students that had 
been largely ignored in the past. The Ministry spent 
$5.3 million on this initiative in the 2010/11 school 
year and has allocated another $1.3 million for 
2011/12. However, no ministry funding has been 
committed for future years. One of the boards we 
visited expressed concerns whether it could sustain 
this program if ministry funding ceased. 

Cooperative Education 

The Cooperative Education program allows stu- 
dents to earn secondary school credits through a 
job placement. In 2005, the program was modified 
so that up to two cooperative education credits 
could be counted as compulsory credits, and 
the program was promoted to students at risk. 
Cooperative education placements are available in 
many kinds of work settings, such as hair styling, 
auto mechanics, television broadcasting, municipal 
government, and nursing. The program is intended 
to complement academic requirements and prepare 
students for the future by providing practical work 
experience. In the 2008/09 school year, using 
the most recent available data, there were 72,000 
students enrolled in cooperative education who 
earned 150,000 credits. 

For a student to earn a cooperative education 
credit, which has the same value as any other 
credit, the job placement must be related to a 
ministry-approved course that the student is 
enrolled in or has completed successfully. A student 
can earn up to two work placement credits for each 
subject credit. In addition to the work experience 
hours a student is required to achieve, students are 
expected to complete a minor classroom component 
that is designed to relate the placement experi- 
ence to the curriculum expectations of the related 
course. Students can earn all 12 optional and two 
of the 18 compulsory credits required to obtain an 



Ontario Secondary School Diploma through the 
Cooperative Education program. 

Over the years, the Cooperative Education pro- 
gram has been promoted as potentially helpful to 
students who are disengaged, returning to school, 
or experiencing developmental delays. All of the 
boards we visited had some form of centralized 
cooperative education program in place. 

Generally, co-op teachers are responsible for 
interviewing students for work placements, finding 
students suitable jobs, and evaluating performance. 
In collaboration with the students, these teachers 
write out the skills students are expected to learn 
at their placements. However, many of the related 
reports we reviewed did not clearly document the 
link between the job placement and the course 
expectations. In a number of cases, students had 
earned or were earning credits for working in a 
wide range of placements, such as clothing stores, 
fast-food outlets, coffee shops, grocery stores, 
municipal planning offices, television studios, and 
laboratories. In many of these cases, we could not 
assess the merit of the work placements reviewed 
and whether the placement complemented the 
in-class experience. In addition, the Ministry and 
boards informed us that a formal analysis has not 
been performed to assess the overall suitability of 
co-op placements in ensuring that students acquire 
the expected knowledge and skills. 

Credit Recovery 

The Credit Recovery program is designed to 
increase student retention by enabling students who 
have failed a course to earn the credit by repeating 
only those course expectations where the student 
had been unsuccessful. Since student performance 
in the earlier secondary school years is considered 
critical to future learning, credit recovery is gener- 
ally directed to grade 9 and grade 10 students. In 
the 2008/09 school year, more than 17,000 students 
received 23,600 credits through credit recovery. 

When the Ministry introduced Credit Recovery 
in 2005, it issued a series of memos to guide 
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implementation. In 2010, the Ministry consoli- 
dated these guidelines in a formal policy docu- 
ment. However, we found these guidelines to be 
general and to provide little specific direction to 
school staff. For example, there are no guidelines 
on the maximum number of optional or com- 
pulsory courses a student could recover and no 
minimum percent a student should have received 
in the original course to be eligible for the Credit 
Recovery program. The Ministry and boards 
informed us that the guidelines are intended 
to be flexible to allow for individual student 
circumstances. 

As a result, we found wide variations in the 
way the Credit Recovery program was delivered. 
According to ministry guidelines, a student's evalua- 
tion can be based solely on performance in the 
credit recovery portion of the course or by merging 
the credit recovery mark with that of the original 
course. In the schools we visited, we found that the 
weight assigned to a student's performance in credit 
recovery relative to the original course mark ranged 
from 30% to 70%. In other words, in one school the 
credit recovery work was worth 70% of the student's 
final mark, whereas in another it was worth only 
30%. No documented rationale was provided for 
the percentage of the student mark awarded for the 
credit recovery portion of the course. 

The subject teacher is required to complete 
a student credit recovery profile indicating the 
units, concepts, and expectations not successfully 
completed. The profile is to be used by the credit 
recovery teacher to develop a learning plan that 
should identify the expectations to be covered, the 
appropriate teaching strategies, and how the final 
mark will be determined. 

We reviewed credit recovery documentation at 
a number of schools and found that many profiles 
and plans were not on file, and those that were on 
file failed to indicate clearly the course expectations 
that had not been successfully achieved and/or the 
expectations to be realized in credit recovery. We 
also found examples where it seemed unclear what 
work students had performed to pass the course 



through credit recovery. In one case, a student who 
had received 24% in a course recovered the credit 
by completing five expectations. This student had 
failed 26 of the 31 original course activities, earned 
a zero in 19 of these activities, and failed the final 
exam with a mark of 14%. Due to the lack of docu- 
mentation, we could not assess whether the student 
met the required expectations for the course. 

In 2010, the Ministry initiated a study at five 
school boards to assess the Credit Recovery pro- 
gram to ensure that students acquired sufficient 
knowledge to be successful at the next level. The 
study examined the subsequent performance in 
grade 10 of students who failed a grade 9 course 
but subsequently passed the course through credit 
recovery or by repeating it. Significant differences 
were noted among the boards in the grade 10 pass 
rates of credit recovery students versus those who 
repeated the grade 9 course in its entirety. How- 
ever, the study ended without drawing conclusions 
because of concerns with small sample sizes and 
incomplete data. To address these concerns, the 
Ministry plans to perform a more comprehensive 
province-wide analysis of the program. 

Specialist High Skills Major 

Introduced in 2006, the Specialist High Skills 
Major (SHSM) program allows students to focus 
their learning on a specific economic sector while 
meeting the requirements to graduate from 
secondary school. SHSM enables students to 
gain knowledge in various career options such as 
agriculture, aviation, business, transportation, and 
mining. The program also helps students prepare 
for the transition to apprenticeship, college, uni- 
versity, or the workplace. In the 2009/10 school 
year, nearly one-half of Ontario's 900 secondary 
schools offered nearly 750 SHSM programs to 
more than 20,000 students. 

School boards submit SHSM applications to 
the Ministry for funding approval. As part of its 
monitoring processes, the Ministry requires school 
boards to submit SHSM student data reports three 
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times a year. The reports include enrolment, num- 
ber of credits attempted and earned, and total num- 
ber of students earning a diploma with an SHSM 
designation. Based on these reports, the Ministry 
prepares a summary report that provides boards 
with information on where they stand in these 
respects relative to the province. These reports help 
boards improve their programs. Also, to help evalu- 
ate and refine the program, Ministry officials regu- 
larly meet with SHSM teachers across the province. 

Overall, the Ministry has put some good mon- 
itoring procedures and a process in place to evalu- 
ate the success of the SHSM initiative. However, 
the current reporting requirements focus on par- 
ticipation, retention, and credit accumulation rates 
because information regarding student destinations 
after graduation is not readily available through 
the current information systems. The Ministry 
informed us that, to better assess the success of the 
SHSM program, the common identifier number 
that it planned to implement in 2012 would help 
track students' post-secondary pursuits. 

To assess the success of the SHSM program, in 
November 2010, the Ministry initiated a survey 
of former SHSM students. For the most part, the 
results of the survey were positive. Six months 
after graduating, nearly two-thirds (64%) of SHSM 
students were registered in a post-secondary pro- 
gram (31% in university, 27% in college, and 6% 
in an apprenticeship), and nearly 70% of students 
declared that the program influenced their career 
and educational plans. 

Dual Credit 

Introduced in 2006, the Dual Credit program allows 
students, while they are still in secondary school, 
to take college or apprenticeship courses that count 
toward both their Ontario Secondary School Dip- 
loma and a post-secondary diploma or apprentice- 
ship certificate. These ministry-approved courses 
are delivered by publicly funded Ontario colleges. 

Dual Credit programs are intended to assist 
secondary school students in their progress toward 



graduation and in making successful transitions to 
college or apprenticeship. The Ministry informed 
us that the focus of the program was two groups: 
disengaged students with the potential to succeed 
and returning students who had left high school 
before graduating. For the 2009/10 school year, the 
Ministry reported that 46% of dual credit students 
were identified as disengaged or as having previ- 
ously dropped out of high school. In 2010/11, there 
were almost 13,000 students enrolled in over 400 
dual credit programs. 

The Dual Credit program is co-funded by the 
Ministry of Education and the Ministry of Train- 
ing, Colleges and Universities and managed by 
the Council of Ontario Directors of Education. 
Regional planning teams are responsible for the 
delivery of the program. There are 16 teams across 
the province, with each team consisting of college 
faculty and high school teachers. On behalf of their 
schools, boards submit program applications to 
their respective regional planning teams. The Min- 
istry, in conjunction with the Ministry of Training, 
Colleges and Universities and the Council, reviews 
the applications submitted by the teams and makes 
recommendations for final funding approval. The 
Council is responsible for administering the funds 
to the regional planning teams and monitoring how 
the money is spent, while the Ministry monitors the 
success of the program. 

The regional planning teams submit student 
data reports to the Ministry twice a year. The prov- 
incial roll-up of these student data reports includes 
information such as the distribution of students by 
age and gender, the number of students considered 
disengaged and underachieving, how many have 
previously dropped out of high school, and the 
retention and success rates of the students who 
participated in the program. The Ministry has also 
collected anecdotal information from administra- 
tors regarding lessons learned and the challenges 
they have encountered with the implementation of 
the Dual Credit program, in addition to students' 
perceptions of its benefits and challenges. 
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In the 2009/10 school year, the Ministry began 
requiring the regional planning teams to prepare 
SMART (specific, measurable, attainable, realistic, 
and timely) goals for the coming year. The Ministry 
performs annual monitoring visits with the teams 
to follow up on the status of the goals from the prior 
year and to learn about the challenges and successes 
each team has had with implementing the program. 
Also, as part of the visits, the Ministry provides 
regional planning teams with a data package that 
includes a three-year comparison of statistics such 
as participation rates, retention rates, and success 
rates in comparison to the provincial average. 

In general, we noted that the monitoring pro- 
cesses in place for the Dual Credit program are far 
more extensive and comprehensive than for any of 
the other Student Success programs and in many 
ways serve as a best practice standard for other 
ministry initiatives. Although the Ministry has not 
evaluated the program to determine if participat- 
ing students are making a successful transition to 
post-secondary schooling, it did conduct a survey to 
determine the status of dual credit students six months 
after leaving the secondary school system. For the 
most part, the results were positive. Almost two- 
thirds (65%) of the dual credit students were regis- 
tered in a post-secondary program (6% in university, 
50% in college, and 9% in an apprenticeship), and 
77% of respondents declared that the program influ- 
enced their career choice and educational plans. 

Student Success School Support 

In 2008, the Ministry introduced the Student Suc- 
cess School Support initiative targeting a limited 
number of schools in boards that had a significant 
number of secondary schools where student 
achievement was below the provincial standard. 
The Ministry informed us that the focus was on 
boards that could make a significant contribution 
toward meeting the provincial graduation target. 
Each participating board received ministry funding 
for an additional student success leader to monitor 
the initiative, and each principal at the selected 



schools was assigned a mentor for support. The 
initiative provided funding to 27 schools in three 
boards in 2008/09, 67 schools in seven boards in 
2009/10, and 85 schools in 14 boards in 2010/11. 

In order to identify low-performing schools, the 
Ministry used student success indicator data from 
grades 9 to 12. Some of the key indicators included 
credit accumulation rates, compulsory and optional 
course pass rates, and province-wide EQAO test 
scores. Based on these indicators the Ministry iden- 
tified 170 of the lowest-performing schools. 

We reviewed the selection of schools for this 
initiative in the 2009/10 school year and noted that 
although three-quarters of the schools selected for 
funding were in the lowest-performing category, 
more than 100 of the lowest-performing schools 
received no funding under this initiative in the 
2009/10 school year. 

As part of this initiative, funded schools are 
required to develop an annual School Support Plan. 
The plan is expected to include SMART goals related 
to underperforming students. In addition, these 
plans are to set out the strategies to achieve the 
plan's goals as well as expected student outcomes. 
For example, one school had a goal to increase the 
grade 9 applied mathematics pass rate by five per- 
centage points from 70.2 to 75.2 over the 2010/11 
school year. It targeted 36 students and focused on 
mathematics reading comprehension, vocabulary 
development, and communicating mathematical 
concepts. To monitor progress, schools are expected 
to report to the Ministry six times a year. 

To determine the initiative's impact on school 
performance, the Ministry compiled data on 
credit accumulation at schools and found that the 
27 participating schools in the first year increased 
their grade 9 credit accumulation by 6.8% and 
their grade 10 credit accumulation by 5.6% over 
two years. However, the Ministry performed its 
analysis on an overall school basis and did not 
have sufficient information to assess the suc- 
cess of specific students who underperformed. 
Consequently, the Ministry could not determine 
whether the initiative was successful in improving 
student achievement in the target group. 
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RECOMMENDATION 3 



To ensure that student success initiatives increase 
the number of students who obtain their Ontario 
Secondary School Diploma and are adequately 
prepared for college, university, apprenticeship, 
or the workforce, the Ministry of Education and 
the province's school boards should: 

• assess the re-engagement initiative to deter- 
mine if the benefits that boards had noted 
justify the cost of maintaining the program 
in future years; 

• disseminate best practices or guidance for 
helping students achieve their community 
service hours before graduation; 

• better link work placements in coopera- 
tive education with course expectations to 
ensure that the placements complement the 
in-class experience as required; and 

• assess the Credit Recovery program to deter- 
mine whether students are achieving the 
required course expectations, and consider 
more detailed guidelines to ensure consist- 
ent program delivery across the province. 



| MINISTRY RESPONSE 




The Ministry will assess the benefits of the stu- 
dent success initiatives and programs to ensure 
that they are effective in increasing the number 
of students who graduate and are adequately 
prepared for post-secondary pursuits. Included 
in this work will be the sharing of effective 
practices and guidelines with school boards, as 
well as guidance regarding documentation that 
clearly identifies the linkage between workplace 
experiences and in-class learning. 



STUDENT SUCCESS FUNDING 

Over the past two school years ending in 2010/11, 
the Ministry has provided nearly $245 million 
($130 million in 2010/11) to deliver Student Suc- 
cess Strategy initiatives to help secondary school 



students succeed and graduate with a high school 
diploma. 

Program Funding 

As illustrated in Figure 4, two of the Student Suc- 
cess programs — Re-engagement and Student Suc- 
cess School Support — are funded based on student 
needs. For these programs, the Ministry allocated a 
higher proportion of funding for lower-performing 
schools and boards. Such needs-based funding 
provides resources to the areas where it is most 
required. Over the past two school years, 2009/10 
and 2010/11, $15 million of Student Success fund- 
ing was distributed based on the Ministry's assess- 
ment of student needs. The remaining $230 million 
was allocated based on student enrolment or based 
on applications submitted by school boards. 

Under enrolment funding, each board is pro- 
vided with the same amount per student rather 
than allocating a greater amount to the boards that 
have a higher percentage of students who need 
additional help. As a result, such a per student 
approach does not focus scarce resources on the 
highest priorities that have been identified. 

Similarly, application-based funding is not based 
strictly on need but is based on board estimates of 
the number of students to be enrolled in the pro- 
grams the boards have in place or are proposing to 
implement. Application-based funding can be a bet- 
ter representation of student needs than enrolment 
because these programs are primarily developed 
for students at risk of not graduating. However, 
application-based funding is still dependent on 
whether the schools and boards take the initiative 
to put programs in place to assist students in need 
of additional supports. 

Although much of the funding for enrolment- 
and application-based programs will ultimately be 
used to support board-identified students at risk, 
overall, it would be prudent to target an increased 
proportion of funding to the school boards that 
need the most assistance. We noted that based 
on various indicators, there was a wide variation 
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Figure 4: Student Success Payments to School Boards ($ million per school year) 

Source of data: Ministry of Education 



Program 1 


Primary Funding Basis 


2009/10 




Student Success Grants 2 


enrolment 


59.6 


60.5 


math and literacy 3 


enrolment 


6.9 


7.6 


other enrolment-based programs 


enrolment 


5.1 


4.8 


Total Enrolment-based Funding 




71.6 


72.9 


Dual Credit 


application 


17.2 


25.8 


Specialist High Skills Major 


application 


21.1 


18.6 


other application-based programs 


application 


1.4 


1.2 


Total Application-based Funding 




39.7 


45.6 


Student Success School Support 


needs 


5.2 


4.4 


Re-engagement initiative 


needs 





5.3 


Total Needs-based Funding 




5.2 


9.7 


Total All Student Success Programs 




116.5 


128.2 



1. There is no separate transfer payment amount provided to school boards related to the Cooperative Education and Credit Recovery initiatives. 

2. Student success grants are not targeted for any specific purpose but must be spent by boards to assist students at risk of not graduating. 

3. Math and literacy funding is primarily provided for professional learning supports for mathematics teachers. 



in student needs between boards. For example, 
when considering grade 10 credit accumulation, a 
key early indicator of student graduation success, 
the percentage of students who were on target 
for graduation ranged from 49% to 92% at the 70 
boards with secondary schools. 

A further analysis of Ontario board program 
funding and credit accumulation also revealed little 
correlation between student needs and funding 
received. For example, a board where 81% of its 
students were on track to graduate received $240 in 
student success funding per student while a second 
board in the same region, with only 69% of its stu- 
dents on track, received only $98 per student. Most 
of the funding disparity between the two boards was 
due to different degrees of board participation in 
programs funded through applications. Given that 
other indicators showed similar anomalies, overall 
funding was often not targeted to the boards with 
proportionally more students in need of support. 

Financial Administration 

After reviewing the Student Success financial 
processes and procedures at the Ministry and at 
the school boards we visited, our audit found that 



the boards accounted properly for funds received. 
In general, the Student Success funding was seg- 
regated into separate accounts to help ensure that 
funds allocated to each initiative were spent on 
those programs. Overall, there were generally good 
processes in place to monitor the transfer payments 
to these school boards to help ensure that funds 
were spent for the purposes intended or carried 
over for these purposes to subsequent years. How- 
ever, we noted some concerns: 

• The Ministry could improve its monitoring of 
board expenditures. For example, we found 
that some board reports on how funding was 
spent were based on budgeted rather than 
actual expenditures. Also, the Ministry did not 
require the boards to submit any evidence of 
expenditures that could be subject to periodic 
verification or certification. 

• In the past two school years ending in 
2010/11, Ontario school boards received 
nearly $8 million in unexpected funding for 
Student Success initiatives late in the school 
year that had to be spent by the end of the 
school year. Such payments make it difficult 
to effectively and efficiently use funds to 
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address the specific needs of students. Each 
of the boards we visited welcomed the funds 
but noted the challenges of finding ways 
to spend them wisely on such short notice. 
Consequently, for programs like the Specialist 
High Skills Major, some boards purchased 
items that schools did not necessarily need at 
the time, such as more tools and equipment. 

• Two application-based programs, Specialist 
High Skills Major (SHSM) and Dual Credit, 
receive funding based on projected enrol- 
ments. However, officials in both programs 
greatly overestimated student participation. 
Consequently, the programs were overfunded 
by $3.1 million in the 2009/10 school year. 
For SHSM, nearly one-quarter of the boards 
overestimated their enrolment by more than 
50% of actual student participation while 
one-quarter of the Dual Credit regional plan- 
ning teams over-projected by more than twice 
the actual enrolment. We observed that one 
of the boards we visited worked closely with 
its SHSM schools to come up with a realistic 
enrolment projection that resulted in a fore- 
cast that was off by less than 7%. In the case 
of the SHSM program for the 2010/11 school 
year, the Ministry advised boards that there 
would be adjustments to funding based on 
substantial differences between the actual 
and projected student numbers. For the Dual 
Credit program, overpayments were to be 
deducted from subsequent years' funding. 

• In addition to enrolment projections, SHSM 
program funding is based on a step-down 
model that allocates more money in the 
early years of the program. The logic for this 
approach is to provide up-front funds for 
materials and equipment to get the program 
started. However, several boards informed 
us that some programs such as construction 
are more costly and more capital intensive to 
run on a continuous basis while others cost 
much less to operate. Some boards we visited 
expressed concerns over the sustainability of 



their programs as they need to update equip- 
ment and other materials to stay relevant. 

• For the Dual Credit program, we found that 
the regional planning teams applied different 
funding mechanisms to distribute money to 
their respective boards and colleges. Some 
teams worked through the boards to deter- 
mine appropriate funding, whereas others 
worked directly with the schools. Funding 
was benchmarked at a maximum of $200 per 
student at the board level and $750 at the 
college level. At the two boards we visited 
that delivered the program, we found that 
teams disbursed the benchmark amounts 
rather than the actual expenditures incurred, 
which were often less. Due to this and other 
program issues, the Dual Credit program was 
overfunded by more than $4.3 million in the 
2009/10 school year. 

• Student Success School Support initiative 
funding that was to be used by school boards 
for professional learning days and effective 
instruction purposes was overfunded by 
almost $2 million in the 2009/10 school 
year. One board we visited informed us that 
it could not spend all of its program funds 
because it could not use all the allotted profes- 
sional learning days. In addition to regular 
professional development days, the board 
considered it excessive to provide 137 more 
days of principal/ teacher time for professional 
learning associated with the Student Success 
School Support initiative. The Ministry noted 
that, for subsequent years, the allotment for 
professional development was to be consider- 
ably reduced. Another board we visited did 
not receive any Student Success School Sup- 
port funding and was not even aware of the 
program. Yet, 12 of this board's 18 secondary 
schools were either low-performing or among 
the lowest-performing schools in the province. 

• In addition to Student Success Strategy fund- 
ing, boards received approximately $140 mil- 
lion in "demographic funding" for secondary 
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schools in the 2010/11 school year. This 
funding is based on school profiles of social 
and economic indicators associated with high- 
risk students. Among the indicators are low 
income, recent immigration, lack of parental 
education, and single-parent status. However, 
we found that the allocations to school boards 
were based on outdated information, as much 
of the source data were derived from the 1991 
and 1996 censuses and Statistics Canada 
information. In the 2010/11 school year, 
school profiles were updated with 2006 data, 
but funding re-allocations were to be phased 
in over four years to give school boards time to 
alter their programs and supports to account 
for the new funding levels. As a result, much of 
the funding was in effect still based on 15- to 
20-year-old data. Finally, the boards visited 
indicated that there was no specific reporting 
to the Ministry on the use of demographic 
funding. 

• The Ministry's contract with the Council of 
Ontario Directors of Education to deliver the 
Dual Credit program requires the Council 
to provide externally audited financial 
statements. However, our audit found that 
a financial adviser completed the Council's 
financial reports. Therefore, the Ministry 
lacked professional assurance that the 
$17.2 million provided to the Council in the 
2009/10 school year had been spent for the 
purposes intended. (The Ministry subse- 
quently requested that the Council submit 
audited financial statements.) In addition, 
the Ministry and the Ministry of Training, 
Colleges and Universities provided the Coun- 
cil with a total of $700,000 in the 2009/10 
school year for program delivery ($335,000) 
and administrative costs ($365,000), but the 
Council did not report back on how these 
funds were used. We also found that most of 
the program delivery funding was paid by the 
Council to consultants for implementation 
advice and guidance to regional planning 



teams, colleges, and school boards, at a cost 
of $500 per day. Although the Council was to 
start up the Dual Credit program, the Ministry 
was to eventually take over the administrative 
responsibilities. Therefore, it may now be 
financially prudent for the Ministry to deliver 
the program itself. 



RECOMMENDATION 4 



To ensure that Student Success Strategy fund- 
ing is spent efficiently to address the specific 
needs of students at risk of not graduating, the 
Ministry of Education and the province's school 
boards should: 

• adopt funding methods that target more 
money for schools and boards where stu- 
dents at risk most need the assistance and 
work with the boards and schools to better 
estimate student participation in application- 
based programs; 

• improve existing processes to monitor board 
expenditures and ensure that overfunding is 
properly accounted for; 

• allocate demographic funding based on the 
most recent data available; and 

• assess the cost and benefits of ministry deliv- 
ery of the Dual Credit program. 

| MINISTRY RESPONSE 

The Ministry agrees that program funding 
must be spent efficiently and will continue to 
work with school boards to ensure that funding 
effectively reaches students deemed at risk of 
not graduating and to improve estimates of pro- 
gram participation where necessary. The Min- 
istry also uses enrolment- and application-based 
funding approaches to support boards' efforts to 
increase student achievement and reduce gaps 
in achievement among students. The Ministry 
will continue to work with school boards to 
ensure that funding is properly accounted for. 
The Ministry will assess the feasibility of deliv- 
ering the Dual Credit program internally. 



Chapter 3 



Ministry of Community and Social Services 

Section 

1 3.14 Supportive Services for 

People with Disabilities 



Background 



Figure 1: Supportive Services Expenditures, 2010/11 

Source of data: Ministry of Community and Social Services 



The Ministry of Community and Social Services 
(Ministry) funds a variety of supportive services 
programs designed to help people with develop- 
mental disabilities live at home, work in their 
communities, and participate in a wide range of 
activities. 

Transfer payments for supportive services 
totalled approximately $571 million in the 2010/11 
fiscal year, an increase of approximately 68% from 
the 2000/01 total of $340 million. This represents 
an average annual increase of approximately 5% 
over the last 10 years. Of the $571 million the Min- 
istry spent in 2010/11, it disbursed $472 million, 
or approximately 83% of the total, through 412 
contracts with transfer-payment agencies in nine 
regions. These agencies provided services to about 
134,000 eligible people. The Ministry-administered 
Special Services at Home (SSAH) program received 
$99 million to serve 24,000 families. The break- 
down of funding is illustrated in Figure 1. 

Agencies that receive transfer-payment funding 
provide or arrange for such services as assessment 
and counselling, speech and language therapy, 
behaviour intervention therapy, and respite care. 
The SSAH program provides direct funding to 
families that have eligible people with disabilities 



Special Services 
at Home (17%) 
[Ministry-administered] 



Passport (5%) 
[agency-administered] 




other services (78%) 
[agency-administered] 



living at home. This money is to be used for pur- 
chasing supports and services beyond those typ- 
ically provided by families and that are designed 
primarily to enhance personal development and 
growth and provide family relief through respite 
care. As well, the agency-administered Passport 
program — a recent Ministry initiative — provides 
direct funding to families for such things as per- 
sonal development, as well as social and recrea- 
tional activities. 
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Audit Objective and Scope 



recommendations and the actions taken by the 
Ministry in planning our audit. 



The objective of our audit of supportive services 
was to assess whether the Ministry of Community 
and Social Services (Ministry) had adequate poli- 
cies and procedures for ensuring that: 

• quality supportive services were provided 
in compliance with legislative and program 
requirements and performance expectations; 
and 

• transfer payments were satisfactorily con- 
trolled and commensurate with the amount 
and value of services provided. 

Senior management reviewed and agreed to our 
audit objective and associated audit criteria. 

Our audit included a review and analysis of rel- 
evant files and administrative policies and proced- 
ures, as well as discussions with appropriate staff at 
the Ministry's head office and four regional offices 
that we visited (Kingston, Ottawa, Sudbury, and 
Toronto) . We also reviewed and analyzed relevant 
files and administrative policies and procedures, 
and we held discussions with senior staff at 13 
transfer-payment agencies within the four regions 
we visited. 

In addition, we met with the Provincial Net- 
work on Developmental Services, which included 
members from a wide range of interest groups, 
such as Ontario Agencies Supporting Individuals 
with Special Needs and the Ontario Association on 
Developmental Disabilities. 

We reviewed several audit reports issued by the 
Ministry's Internal Audit Services, including its 
2006 transformation project review of the SSAH 
program and its 2010 Regional Office Controller- 
ship Review of the South West Regional Office. 
Although these reports did not reduce the extent 
of our work, they did influence our thinking about 
specific issues and the approach to our work with 
respect to those issues. 

We also reviewed the 2008 Deloitte report on 
the Ministry's Passport program, which made a 
number of recommendations. We considered these 




Many of the concerns noted in our last audit of this 
program, which took place 15 years ago, have still 
not been satisfactorily addressed. As a result, the 
Ministry still does not have adequate assurance 
that its service delivery agencies are providing an 
appropriate and consistent level of support in a 
cost-effective manner to people with developmental 
disabilities. 

Specifically, the Ministry's oversight proced- 
ures are still not adequate to ensure that quality 
services are provided and that public funds are 
properly managed by transfer-payment agencies. 
For example, ministry staff rarely visit agencies for 
these purposes. Such visits would be particularly 
important given the inadequate accountability 
mechanisms we noted during our audit. 

The Ministry has for several years been under- 
taking a comprehensive Developmental Services 
Transformation project. When the project is com- 
plete, the Ministry expects to have made a number 
of significant changes to the system of develop- 
mental services and supports. However, given the 
extent and complexity of the changes, it will take 
several years before many of the issues we identify 
in this report can be effectively addressed. 

With respect to ensuring that quality services 
were provided by transfer-payment agencies in com- 
pliance with legislative requirements and program 
policies and procedures, we found the following: 
• In half the cases we reviewed, agencies lacked 
supporting documentation to adequately 
demonstrate a person's eligibility or needs. As 
a result, agencies could not demonstrate, and 
the Ministry could not assess, an individual's 
needs and whether the individual was receiv- 
ing the appropriate level of service or, for 
example, was in need of additional support. 
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• The Ministry has not established accept- 
able standards of service, or the necessary 
processes to properly monitor the quality of 
services provided and whether it is receiving 
value for money for the funding provided to 
community-based agencies. 

• The Ministry is not aware of the number of 
people who are waiting for agency-based sup- 
portive services, information that is necessary 
for assessing unmet service needs. 

• Although one would expect a consistent set 
of rules about what are appropriate services 
and, therefore, allowable expenditures under 
the Passport program, the Ministry has not set 
such rules. As a result, expenses for services 
reimbursed in one region were deemed ineli- 
gible for reimbursement in another. In addi- 
tion, claims by individuals under the Passport 
program often lacked the details necessary 

to ensure the appropriateness of expenses 
approved and reimbursed. For example, a 
family was paid a total of $22,000 for a year; 
however, the monthly expenditure reimburse- 
ment requests submitted by the family did not 
provide any information to demonstrate that 
the funding was being requested for eligible 
purposes. 

Our observations with respect to the Ministry's 
oversight of funding provided to transfer-payment 
agencies are as follows: 

• In practice, annual agency funding continues 
to be primarily historically based rather than 
needs-based, and this exacerbates any previ- 
ous funding inequities. As a result, we were 
not surprised to find that some hourly service 
costs appeared excessive and that the range 
of costs per hour for similar services varied 
widely across the province. 

• The quarterly reporting and annual transfer- 
payment reconciliation processes are ineffect- 
ive and serve little purpose given that: 

• information provided by agencies in these 
reports is often not accurate and reflective 
of operations; 



• the quality of the Ministry's review is inad- 
equate and is performed by staff without 
adequate training and expertise; and 

• the Ministry does not adequately review 
the year-end transfer-payment reconcilia- 
tions to ensure that they were properly and 
consistently completed. 

• The Ministry had little knowledge of whether 
the agencies it funded and their boards of 
directors had effective governance and control 
structures in place. We found that, in one 
case, even when serious concerns were identi- 
fied, neither the board nor the Ministry took 
appropriate action. 

With respect to the Special Services at Home 
(SSAH) program, which directly reimburses indi- 
viduals and families for eligible expenses, we found 
the following: 

• Our review of a sample of case files for people 
who received SSAH funding found that the 
forms were properly completed, and in most 
cases people received the support they were 
entitled to under the program's decision 
guide. 

• Since 2008/09, no additional SSAH funding 
has been provided to address the gap between 
the growing demand and available funding. 
As a result, as of March 31, 2011, there was a 
waiting list of almost 9,600 people who met 
the eligibility criteria but were still waiting for 
SSAH funding. 

• We were often unable to determine, and the 
Ministry was unable to demonstrate, that the 
claims submitted and the reimbursements 
made to families were for eligible expenses. 

We also found that the Ministry has not ensured 
that transfer-payment agencies complied with the 
government directive regarding travel, meal, and 
hospitality expenses. We noted a number of pur- 
chases made by senior management at the agencies 
that did not comply with the government directive 
on travel, meal, and hospitality expenses, or with 
good business practices. 
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| OVERALL MINISTRY RESPONSE 

The Ministry appreciates the recommendations 
of the Auditor General. As part of its long-term 
plan to transform the developmental services 
system, the Ministry has already initiated 
several improvements that are consistent with 
the recommendations concerning account- 
ability, eligibility for and access to services, and 
administration. 

In January 2011, the Ministry put in place 
more robust quality assurance measures to set 
service standards for all agencies that receive 
provincial funding for developmental services. 

Subsequent to the audit, the Ministry 
implemented a new way for people to apply for 
services. This addresses the Auditor General's 
concerns about inconsistency in eligibility. Indi- 
viduals will now apply for services based on the 
same criteria through a single streamlined and 
consistent process. 

We are continuing to introduce measures to 
make sure that public funds are managed more 
effectively. We have implemented a stronger 
approach to assess financial risk in agencies 
delivering services and will be introducing new 
reporting standards to improve service quality 
and financial information. 

The Ministry is also improving the admin- 
istration of the Special Services at Home and 
Passport programs. We are moving to a single 
direct funding program to make the system eas- 
ier to navigate and more flexible for individuals 
and families. As of April 1, 2012, Special Servi- 
ces at Home will no longer serve adults with a 
developmental disability. Adults will apply to 
Passport for direct funding; children will apply 
to Special Services at Home. As part of this 
change, the Ministry is reviewing the Passport 
guidelines and will clarify eligible expenses for 
reimbursement under both programs. 




Detailed Observations 



SERVICES PROVIDED BY TRANSFER- 
PAYMENT AGENCIES 

Eligibility and Access to Services 

At the time of our audit, agencies in the nine 
regions, which account for 80% of total program 
expenditures, were using one of two access models 
to enable people to obtain services — either a single- 
agency model or a multi-agency collaborative 
model. 

In a single-agency setup, people apply to an 
agency in the community that has been designated 
as the single access point. This agency performs the 
initial screening, determines a person's eligibility 
for supports and services, and matches his or her 
needs to the available services. In a multi-agency 
collaborative setup, a person or family can apply 
directly to any agency in their community. 

To qualify for and receive supportive services, 
a person is first assessed for eligibility by the 
agency he or she approaches. The agency prepares 
a formal assessment to determine what services 
the person needs. We found that, in most cases, 
the Ministry did not provide guidance to agencies 
regarding the criteria and documentation required 
to demonstrate someone's eligibility and, therefore, 
their needs. As a result, half the cases we reviewed 
lacked documentation supporting the diagnosis 
of the individual's specific disability and needs. In 
the absence of such documentation, the Ministry 
cannot readily ascertain whether service needs are 
determined on a fair and consistent basis through- 
out Ontario and that the services recommended are 
the most appropriate for the individual's needs. 



RECOMMENDATION 1 



To help ensure that eligibility is determined 
consistently and equitably across the province, 
and that individuals receive the appropriate 
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support, the Ministry of Community and Social 
Services (Ministry) should provide guidance to 
agencies regarding the criteria and documenta- 
tion required to demonstrate a person's eligibil- 
ity and needs. The Ministry's regional offices, as 
part of their oversight responsibilities, should 
then periodically review whether transfer- 
payment agencies are assessing people on a 
consistent basis and matching their needs to the 
most suitable available services. 



| MINISTRY RESPONSE 

In July 2011, the Ministry implemented a new 
way for people to apply for developmental 
services and supports. Nine Developmental 
Services Ontario organizations are now the 
single windows through which adults with a 
developmental disability and their families 
apply for services and supports. Decisions about 
eligibility for support are made the same way 
across the province, on the basis of consistent 
criteria. Everyone will be assessed in the same 
way regardless of where he or she lives in the 
province. This new process means that eligibil- 
ity will be determined consistently across the 
province. 



Quality of Services Provided 

Establishing measurable service standards can be 
challenging, given that agencies provide a wide 
range of programs, and that service needs can vary 
significantly from person to person. However, it is 
important for the Ministry to set quality-of-service 
standards to help ensure that programs delivered 
by agencies meet people's needs and ultimately 
represent value for money spent. Common bench- 
marks such as staff-to-client ratios, assessment of 
staff qualifications, and, ultimately, assessment of 
program outcomes are useful tools for evaluating 
the quality and cost-effectiveness of the develop- 
mental support services being provided. 



However, based on our review of program files 
and discussions with ministry and agency staff and 
stakeholder groups, we noted that, similar to find- 
ings from our last audit of this program in 1996, the 
Ministry does not have a set of standards or a pro- 
cess in place for periodically assessing the quality 
of services provided by agencies. As well, we noted 
that the regional offices seldom made on-site visits 
to the various agencies responsible for service deliv- 
ery in their regions to gain first-hand knowledge of 
their operations. As a result, the Ministry cannot 
assess whether agencies have provided the right 
services given the individual's needs or whether 
value for money has been received for the funding 
provided to that agency. 



RECOMMENDATION 2 



To ensure that services are appropriate, are of 
an acceptable standard, and represent value for 
the money spent, the Ministry of Community 
and Social Services should: 

• establish acceptable standards of service; and 

• periodically evaluate the appropriateness 
and cost-effectiveness of the services pro- 
vided by transfer-payment agencies. 

| MINISTRY RESPONSE 

The Ministry monitors service agencies through 
an annual funding agreement that sets out the 
Ministry's expectations and requirements for 
service delivery for each program area. 

In January 2011, the Ministry introduced 
a new regulation that established more robust 
quality assurance measures for agencies. These 
measures are intended to help set consistent 
standards and evaluate the appropriateness 
of the services and supports being delivered 
to adults with a developmental disability. All 
Ontario-funded developmental services agen- 
cies were trained on the measures, and ministry 
staff will regularly follow up with agencies to 
make sure they are complying. 



Supportive Services for People with Disabilities 



The Ministry is also planning to conduct an 
evaluation of the implementation of the new 
Developmental Services Ontario organizations 
as part of its transformation initiatives. These 
organizations provide a single point of access 
for adults with a developmental disability and 
their families to apply for services and supports. 
Anticipated as part of this evaluation is whether 
individuals with a developmental disability 
receive the appropriate services as identified in 
their assessments, as well as an analysis of the 
cost of these services. 

In addition, the Ministry is co-leading a long- 
term human-resource strategy with the develop- 
mental services sector to recruit and retain 
qualified professionals to ensure that there is 
a well-trained, skilled workforce to support 
individuals. A key component of the strategy 
has been the development of core competencies 
for positions at all levels in the sector. These 
core competencies will help improve the skill 
sets of direct support staff as well as of agency 
management. 



Wait Lists 

People who are assessed as eligible for supportive 
services, but for whom agency-based services or 
direct funds are not available at the time of assess- 
ment, are placed on a waiting list. Lack of access to 
supports or services — and the resulting wait lists — 
can arise because of inequitable distribution of 
funds among the regions. As well, some areas of the 
province have limited access to certain professional 
services, resulting in longer waits for such services. 

Agencies within each region maintain, either 
collectively or individually, the wait-list informa- 
tion for applicants who are determined to be 
eligible and in need of service. There currently is 
no standard approach in maintaining waiting lists 
for agency services, and, with the exception of 
the Passport program, wait-list information is not 
provided by the agencies to the Ministry's regional 



offices. Therefore, with the exception of the direct 
funding programs (SSAH and Passport), the Min- 
istry is not aware of the number of people waiting 
for agency-based supportive services, information 
that is necessary for assessing unmet service needs. 
Wait-list information, once collected and analyzed, 
would help the Ministry identify where the need is 
greatest and would help it, for example, distribute 
funding more effectively. 



RECOMMENDATION 3 



To help monitor and assess unmet service needs, 
and help allocate funding more equitably, the 
Ministry of Community and Social Services 
(Ministry) should work with agencies to ensure 
that they prepare and periodically forward to 
the Ministry accurate wait-list information on a 
consistent basis. 

| MINISTRY RESPONSI 

A key goal of Ontario's developmental services 
modernization is improving fairness and equity 
in how funding decisions are made. Develop- 
mental Services Ontario will now be responsible 
for assessing everyone's needs in a consistent 
way. Its work will be supported by new technol- 
ogy that will maintain accurate information 
about service needs and wait lists across the 
province. 

The next step in the modernization plan is 
a new funding approach that will consistently 
prioritize service for people who need it most. It 
will also make funding more equitable by tying 
funds to each person's assessed needs, so that 
people with similar needs receive similar levels 
of support. 



Passport Program 

In the 2005/06 fiscal year, the Ministry imple- 
mented an initiative called the Passport program 
to give an annual block of funding to agencies to 
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be given to families of eligible people who have left 
school or who are waiting for community-based 
services. Under this program, people may receive 
funding to help them get involved in continuing 
education, volunteering, leisure activities, and 
social skills development, as well as to get help with 
employment preparation and vocational activities. 

In the 2010/11 fiscal year, 2,700 people received 
a total of $31 million (or an average of $11,500 
each) in Passport funding. In addition, there 
were approximately 4,500 people who had been 
determined to be eligible, but who, because of the 
limited funding available, were on the Passport 
funding wait list. 

Passport Program Reimbursements 
Once a region's designated Passport program 
agency has determined eligibility and approved 
funding amounts, clients and their families can 
choose either to receive funding directly to pur- 
chase services themselves or to have the agency 
administer the funding on their behalf. Families 
that choose to receive funding directly must submit 
detailed invoices to the Passport-designated agency 
for approval and reimbursement. 

Our review of a sample of claims found that the 
process for ensuring that funding was spent only 
for eligible services was ineffective for the following 
reasons: 

• The Ministry has not set out clearly what 
are appropriate uses of Passport funding. 
As a result, expenditures being approved 
in one region were not deemed eligible for 
reimbursement in another. For example, 
reimbursements for entertainment expenses 
at times included expenses for the support 
worker only, at other times for both the 
support worker and the client, and at other 
times the support worker, the client, and 
accompanying friends and family members, 
depending on which region the client was in. 

• There is inadequate control by agencies over 
the review and approval of reimbursements. 



All files we reviewed had instances where the 
invoices lacked normally expected informa- 
tion such as specific dates, what activity 
was being claimed for reimbursement, and 
the duration. For example, a family was 
reimbursed $22,000 for a year with monthly 
invoices that simply noted "volunteer job 
activities," "health and fitness in the com- 
munity," and "personal skills development." 
Another client was reimbursed for a $7,000 
invoice that listed only "recreation" activities 
for an 11-month period. 



RECOMMENDATION 4 



To ensure that families are being reimbursed 
only for the reasonable cost for eligible activ- 
ities, the Ministry of Community and Social 
Services should clearly define what are eligible 
expenditures and ensure that agencies are 
approving and reimbursing expense claims on a 
consistent basis across the province. 



| MINISTRY RESPONSE 




The Ministry is moving toward a single direct 
funding program in April 2012. To prepare for 
this change, the Ministry has begun reviewing 
its guidelines for Passport. The new guidelines 
will specify more clearly the services and 
supports that can be purchased through this 
program and the reporting and accountability 
requirements. 



MANAGEMENT AND CONTROL OF 
TRANSFER-PAYMENT CONTRACTS 

Budget Submissions and Annual Service 
Contracts 

The Ministry enters into annual service contracts 
with each of its supportive services transfer-payment 
agencies. The agencies submit annual budget 
proposals, which are to include details about the 
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amount of program funding they are seeking, and 
the types and quantity of services to be provided. 
The process then calls for the Ministry to review 
the budget submission package to help ensure that 
the final contract entered into provides for quality 
services that represent value for money spent. 

We found that the Ministry's budget review and 
contract approval process does not ensure that the 
approved amount of funding is reasonable and 
commensurate with services to be provided. In the 
cases we reviewed, there was little or no evidence 
that the Ministry had performed any analysis of 
budget submissions. 

This is of particular concern for the following 
reasons: 

• In most of the files we reviewed, there were 
significant variances in budgeted service 
targets and requested funding amounts 
compared to the previous year's approved 
contract. For example, the service target for 
one program decreased by almost half — 
from 39 to 21 individuals — but the funding 
requested by the agency almost doubled — to 
$803,000 from $440,000. The Ministry subse- 
quently approved the contract for the agency 
to receive $840,000 to serve the 21 people. 
Although the total number of people served 
isn't the only indicator of what funding an 
agency should receive, the significant decrease 
in clients served should have warranted 
follow-up questions before an almost doubling 
of the previous year's funding was approved. 



• The service targets and funding amounts on 
the agencies' budget submissions were often 
significantly different from those on the con- 
tract that was ultimately approved, and there 
was no evidence of the Ministry's rationale 
for the approved amounts. For example, for 
one agency the service target for one of its 
programs decreased significantly, from 51 
individuals on the budget submission to nine 
on the approved contract, yet the original 
$79,000 requested for 51 people was not 
changed and was ultimately approved. 
Except for minor adjustments for special initia- 
tives and new programs, service contracts — includ- 
ing service targets and total funding amounts — are 
generally rolled forward from year to year. There's 
no evidence that the Ministry assessed the rea- 
sonableness of the funding approved, vis-a-vis the 
services to be provided. 

The cost per hour for particular types of services 
varies widely among regions. We asked the Ministry 
whether it compares the cost of similar services 
between agencies within each region or across 
the province to determine whether the costs are 
reasonable. We were advised that the Ministry does 
not do such comparisons. We analyzed the cost per 
hour of direct service for various types of programs 
in three of the four regions we visited and noted a 
wide range. Figure 2 shows the cost range for some 
services. 

The costs per hour for different types of sup- 
portive services are expected to vary, sometimes 



Figure 2: Cost Range for Selected Adult Services, 2009/10 

Source of data: Ministry of Community and Social Services 





Cost per Hour of Direct Service 




Province 


Three Regions Visited 




Average ($) 


Average ($) 


Lowest ($) 


Highest ($) 


# of Agencies* 


out-of-home respite care 


61 


20 


4 


457 


35 


assessment and counselling 


86 


121 


10 


487 


32 


day programs 


33 


45 


8 


74 


59 


client case management 


32 


48 


16 


881 


52 



* Number of agencies that are providing each specific service within the three regions from which we obtained the information 
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significantly, depending on the type of service a 
client requires, although the cost per hour of similar 
services should be within a reasonable range. How- 
ever, as seen in Figure 2, some costs per hour appear 
excessive, and the range of costs per hour for similar 
services is extreme. The Ministry does not have the 
information necessary to assess what constitutes a 
reasonable hourly cost or a reasonable range. 

We also noted that a 2%-a-year base funding 
increase was provided to agencies beginning in the 
2007/08 fiscal year up to and including 2009/10. 
This increase was part of the 2007 Ontario Budget 
announcement to enhance services and supports in 
the developmental services sector. 

Because increases such as the 2%-a-year base 
increase were given without any consideration of 
the agencies' prior-year surpluses or deficits, or 
changes in service demands, any previous funding 
inequities were not addressed. We noted similarly 
in our 1996 audit that across-the-board percentage 
funding changes perpetuated historical funding 
inequities. We further noted that there was insuffi- 
cient evidence in the files we reviewed that the 
Ministry related the amount of an agency's total 
funding approval to an assessment of the value of 
the underlying services to be provided or the com- 
parative need for services in that local community. 
For example, the Ministry did not determine the 
cost per unit of service to permit the comparison of 
the costs for similar services or the identification of 
higher-cost services that could benefit from a more 
detailed review. 



RECOMMENDATION 5 



To ensure that funding provided to transfer- 
payment agencies is commensurate with the 
value of services provided and that funding is 
primarily provided based on local needs, the Min- 
istry of Community and Social Services should: 
• reassess its current budget submission, 
review, and approval process and revise it to 
ensure that the approved funding to agen- 
cies is appropriate for the expected level of 
service; and 



• analyze and compare the agency costs of 
similar programs across the province, and 
investigate significant variances that seem 
unjustified. 



MINISTRY RESPONSE 



The Ministry is developing a new funding allo- 
cation model that will improve equity, allocate 
funding on the basis of assessed need, and pro- 
mote cost-effectiveness. 

In the 2012/13 fiscal year, the Ministry will 
introduce new Transfer Payment Reporting 
Standards that will help improve the Ministry's 
ability to compare costs between agencies that 
are providing similar programs. Following 
that, additional financial data standards will 
be implemented that will allow more accurate 
information on program cost factors and vari- 
ances. Ministry and agency staff will be trained 
to ensure a consistent approach to contract 
management and analysis of quarterly reporting 
information. 



Ministry Oversight and Control 

The government transfer-payment accountability 
Best Practices guide states: 

It is not enough to have an agreement 
in place and then file it away. [Transfer- 
payment] program managers have to 
read, understand, and actually use and 
enforce these agreements in managing 
the [transfer-payment] relationship on a 
day-to-day basis. So while an agreement is 
an essential instrument to have in a well- 
managed [transfer-payment] program, it 
is not a substitute for program managers' 
due diligence at every stage of the transfer 
payment accountability cycle. 

To assess whether the Ministry was adhering to 
this directive in monitoring the quality of services 
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and the value-for-money performance of the 
community-based agencies it funds through trans- 
fer payments, we looked at two things. The first 
is the accuracy of the information that the agen- 
cies report to the Ministry, and the second is the 
process that the Ministry has in place for assessing 
that information in relation to the annual agency 
contract, including performance benchmarks. Our 
review found that neither of these requirements 
was fulfilled to a sufficient standard to allow proper 
oversight of community-based service delivery. 
These requirements are particularly important in 
view of the fact that regional offices do not conduct 
periodic on-site visits of agencies. 
In particular: 

• Although the Ministry requires that agencies 
file quarterly and year-end reports to inform it 
of such things as budgeted expenditures com- 
pared to actual expenditures, and expected 
services being funded compared to actual 
services provided, agencies often did not 
accurately or adequately report key informa- 
tion to the regional offices. 

For example, we found that many agen- 
cies report their service results by replicating 
their approved targets or making arbitrary 
allocations, regardless of actual clients served. 
Almost all of the agencies maintained client 
lists that differed, sometimes significantly, 
from what was reported to the Ministry. 
For example, one agency reported serving 
65 people in its respite-care program when it 
actually served 26. Another reported serving 
25 people in its day program when it served 
194. When asked about the basis for the num- 
bers reported to the Ministry, these agencies 
said that they were arbitrary numbers deter- 
mined in previous years and were reported to 
match the approved funding contract for that 
year. 

We also found that programs' service 
hours and administration costs reported by 
agencies do not represent the actual costs. 



Once again, agencies told us that they arbi- 
trarily allocate those amounts to programs. 

We also found that some agencies did 
not submit the required audited financial 
statements, post-audit management letters, 
or other supporting information to substanti- 
ate expenditures and adjustments on their 
year-end reports, known as Transfer Payment 
Annual Reconciliation (TPAR) reports. 

We recognize that agencies have little 
incentive to report actual service and expendi- 
ture data accurately, since Ministry-approved 
funding amounts are based primarily on 
historical data and are consistently rolled for- 
ward from year to year, regardless of the level 
of actual services being provided. As well, 
in most cases, there are no consequences for 
agencies that report inaccurate or misleading 
results. 

• The Ministry does not have in place adequate 
procedures for reviewing the information 
that it receives from agencies to determine its 
accuracy, or for following up on inconsisten- 
cies even when they're evident. We found 
that the Ministry does not request supporting 
information, such as client lists, in order to 
confirm whether data from the agencies are 
accurate and reflect actual operations; nor, as 
previously noted, do regional office staff visit 
the agencies to gain first-hand information on 
the level of services actually being delivered. 
The Ministry also doesn't confirm whether 
data were reported in accordance with the 
instructions it sends out to the agencies. 

We found no evidence that the Ministry fol- 
lowed up on any of the cases where there were 
significant variances between approved and 
actual reported service targets on quarterly 
reports, even though there was no explanation 
provided by the agencies, or the explanations 
were insufficient. As well, the Ministry did not 
identify and analyze variances in data from 
one quarterly report to another. For example, 
for the first three quarters of the fiscal year, an 
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agency reported serving 15 people in a pro- 
gram; in the fourth quarter, it reported only 
four people in the program. 

We also found that the two ministry units 
that handle the reviews of the quarterly 
and year-end reports, respectively, operate 
independently and therefore do not benefit 
from each other's knowledge of the agencies' 
files. 

When agencies did submit the required 
financial information in their year-end 
TPAR reports, the Ministry did not properly 
reconcile the reports to the agencies' related 
audited financial statements. We reviewed the 
financial information provided in a sample 
of TPAR reports and identified a number of 
inappropriate expenses that the Ministry did 
not identify, but should have. For example, 
we found capital purchases that were made 
using transfer-payment funds approved for 
delivery of supportive services. In half of these 
cases, the agencies reported on their TPARs 
that annual program operating funds of up 
to $540,000 were used for one-time capital 
purchases, the details of which were not docu- 
mented. Ministry staff in this region told us 
that they compared totals rather than doing 
a line-by-line review of the financial informa- 
tion provided. Our scan of the information 
indicated that a line-by-line review would 
have highlighted these unauthorized major 
capital purchases for follow-up. 
We also noted that ministry staff responsible 
for review and approval of financial submissions 
from agencies often did not have the necessary 
training and expertise. As a result, the staff cannot 
effectively review and interpret the information 
from agencies. For example, ministry staff relied 
on the audited financial statements of agencies 
to ensure that transfer payments were spent pru- 
dently and for their intended purposes. However, a 
financial-statement audit isn't intended to provide 
assurance that funds were spent prudently and for 
the intended or eligible purposes; it ensures only 



that what the funds were spent for is accurately 
reported in the agency's financial statements. 



RECOMMENDATION 6 



To ensure adequate oversight of transfer-payment 
agencies and to improve accountability within 
the supportive services program, the Ministry of 
Community and Social Services should: 

• review all agency quarterly reports and 
year-end TPAR submissions for unusual or 
unexplained variances from previous years 
and from contractual agreements, and fol- 
low up on all significant variances; 

• perform spot audits on agencies to validate 
the information provided in the quarterly 
reports and TPAR submissions; and 

• assess whether each regional office has the 
level of financial expertise required, and, 
where lacking, determine the best way of 
acquiring this expertise. 

| MINISTRY RESPONSE 

Introduction of new Transfer Payment 
Reporting Standards in the 2012/13 fiscal year 
and additional financial data standards will 
enhance the Ministry's ability to assess whether 
or not value for money was received and require 
that significant variances be explained. 

Ministry staff will receive additional train- 
ing to support a more consistent approach to 
contract management and analysis of quarterly 
reporting information. 

Work is also under way on two separate 
Transfer Payment Governance and Account- 
ability Frameworks, one for ministry staff and 
one for service providers. The frameworks will 
promote a stronger understanding of ministry 
business practices and risk management to 
improve accountability in the management of 
transfer payments. 

The Ministry's new legislation for develop- 
mental services includes requirements for qual- 
ity assurance measures and allows ministry staff 
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to conduct site visits with agencies. These visits 
can include inspection of financial records. 



Governance and Accountability 

Agencies that receive transfer payments are 
required to have effective governance structures 
and accountability processes in place to properly 
administer and manage public funds. 

However, contrary to what one would expect, 
especially for agencies receiving significant funding, 
the Ministry had little knowledge of whether agen- 
cies and their boards had the expertise and experi- 
ence necessary to discharge their responsibilities in 
compliance with ministry requirements, and had the 
appropriate governance and control structures. 

Although smaller agencies receive less funding, 
appropriate oversight is still critical, especially 
because separation of duties is inherently difficult 
at small agencies, which are often run by a single 
individual. Generally, such agencies have insuffi- 
cient resources to achieve the proper segregation 
of duties found in larger organizations. Although 
the primary oversight role rests with the boards of 
directors, the Ministry still needs to be cognizant of 
the risks, so any concerns identified to the boards 
and the Ministry should be addressed promptly. 

However, we found examples where even when 
concerns were identified, neither the board of dir- 
ectors nor the Ministry took action. In one agency 
we visited, we noted that the executive director 
performed all the accounting functions and was the 
only person who had access to the agency's finan- 
cial information, such as bank records and journal 
entries. This agency's external auditors noted in 
their report to the board of directors that errors and 
omissions in the agency's financial records resulted 
in internal financial statements that differed mater- 
ially from the actual financial position and results 
of the agency's operations. The external auditors' 
report also highlighted concerns over the conflict- 
ing duties of the executive director. Subsequent to 



this report, the board of directors fired the external 
auditors and appointed new auditors. The Ministry 
also obtained the report from the auditors but did 
not question or even follow up with the board or 
the agency. 

We also identified a number of questionable 
expenses at larger agencies, such as retirement 
gifts and frequent staff appreciation meals. At 
one agency, when we brought such examples of 
inappropriate expenditures to the attention of 
senior executives, the response from one was that 
the agency would simply charge those expenses 
to a different account in the future, so as not to 
raise any suspicion in upcoming audits. Our sense 
was that senior management did not appear to 
understand that the account to which the expenses 
were charged was not the issue; rather, it was the 
questionable use of taxpayer money. 

Based on the findings in this report and our dis- 
cussions with ministry and agency staff, we believe 
that the Ministry's oversight procedures are not 
adequate to ensure that public funds are well spent 
and properly managed by agencies and their boards 
of directors. 



RECOMMENDATION 7 



To ensure that agencies have the capabilities 
to properly administer the spending of public 
funds, the Ministry of Community and Social 
Services should encourage the regional offices 
to play a more hands-on role in ensuring that 
agencies have appropriate expertise and govern- 
ance structures and accountability processes, 
including those smaller agencies that receive 
less funding but may have more difficulty main- 
taining proper financial controls. 

| MINISTRY RESPONSI 

The Ministry is committed to strengthening 
governance and accountability in the use of 
public funds. Work is under way on two separate 
Transfer Payment Governance and Account- 
ability Frameworks, one for ministry staff and 
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one for service providers. The frameworks will 
promote a stronger understanding of ministry 
business practices and risk management to 
improve accountability in the management of 
transfer payments. The Ministry is also refining 
its risk assessment tools, introduced in 2008, for 
fall 2011. Ministry staff use the tools to assess a 
broad range of risks, including those associated 
with governance and accountability. 



SPECIAL SERVICES AT HOME (SSAH) 

Under its Special Services at Home (SSAH) pro- 
gram, the Ministry directly funds, at an average of 
$4,200 each, 24,000 individuals or families that 
have elected to manage the services for an eligible 
adult or child with a developmental disability, or 
for a child with a physical disability. The funding 
provided is intended to assist the eligible individual 
and his or her family in purchasing services such as 
family relief, or for personal growth and develop- 
ment for developmentally disabled individuals. 

In the 2008/09 fiscal year, the Ministry decided 
to freeze SSAH funding while it looked at ways 
to address the gap between the growing demand 
and available funding. Since this freeze came 
into effect, no additional individuals have been 
approved for funding, resulting in a wait list of 
almost 9,600 people who had been determined to 
be eligible and were waiting for SSAH funding as of 
March 31, 2011. 

Eligibility for SSAH funding is restricted to 
adults and children with developmental disabilities 
or children with physical disabilities, provided that 
they are residents of Ontario, have ongoing func- 
tional limitations as a result of their disabilities, 
require support beyond that which is typically 
provided by families, and are living at home with 
their families or are living outside the family home 
but do not receive residential staff support from a 
government-funded source. To qualify, a person 



must have written documentation from a physician 
or psychologist that outlines his or her disability. 

To help regional offices provide funding com- 
mensurate with an applicant's needs and to ensure 
that levels of funding are comparable for people 
with similar needs, the Ministry in 2004 imple- 
mented the Decision Support Guide. The guide 
includes 15 questions to be used by ministry staff 
to assess the level of a person's needs on a point 
system in eight major categories. The accumulated 
score for the 15 questions then determines the 
maximum amount of funding for which the person 
is eligible. 

Our review of a sample of case files for people 
who received SSAH funding found that the forms 
were properly completed, and in most cases people 
received the support they were entitled to under 
the decision guide. 

However, there were many cases in which there 
were changes to an individual's decision-guide 
score from one year to the next — something that 
could change the amount of funding for which 
he or she would be eligible. Even in those cases 
where the change in score did change the funding, 
there was no additional information to support the 
change in score. For example, one person whose 
score changed from one year to the next without 
any documented rationale received a funding 
increase of $4,000, or 66%, from the previous 
year's funding. The increase was approved while 
the SSAH funding freeze was in effect. 

SSAH Reimbursements 

SSAH funds help eligible people and their families 
purchase support services that would otherwise 
not be available to them. These must be for one of 
two purposes: to help with the client's personal 
development and growth or for the family's relief 
and support, including respite care. 

There are a number of services available in the 
community, and families are expected to bear some 
costs, regardless of their situation. Therefore, there 
are services that are not recognized or funded 
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through SSAH, including basic care, child-care fees, 
assistive devices and specialized equipment, dental 
services, and home modifications. Although the 
Ministry has produced a list of ineligible expenses 
that will not be reimbursed, it has not defined pre- 
cisely what expenses do qualify for SSAH funding. 

Individuals may choose to purchase services 
themselves with their approved SSAH funding or 
may elect to have an agency administer the fund- 
ing on their behalf for a negotiated administration 
fee. In either case, for individuals and families 
to recover expenses incurred under the SSAH 
program, they must submit invoices that are sup- 
ported by appropriate documentation either to 
the regional office if they self-administer, or to the 
agency they designated to administer their funds on 
their behalf. 

Our review of submitted claims and reimburse- 
ments paid directly to families by the Ministry or 
through an agency found that there was inadequate 
information and review of reimbursement claims 
to ensure that payments met the intent of the pro- 
gram. Following are some examples: 

• In some cases, claims were inappropriately 
approved and reimbursed for such things as 
basic care by the primary caregiver and for 
duplicate invoices. For example, two identi- 
cal invoices of $4,100 submitted in the same 
month by a family were approved and reim- 
bursed without question by the Ministry. 

• An invoice for $4,560 was reimbursed, 
although detail that should be expected, such 
as specific dates the service was rendered, 
and the hours and rate charged by the person 
providing the service, were missing. 

We also found that in the small number of cases 
in which individuals elected to have an agency 
administer the funding on their behalf, the Ministry 
neither requested invoices from the agencies to 
substantiate the SSAH reimbursements nor per- 
formed any spot audits to verify amounts claimed 
by agencies. 



RECOMMENDATION 8 



To ensure that Special Services at Home (SSAH) 
reimbursements to families are consistently 
made only for legitimate and eligible expenses, 
the Ministry of Community and Social Services 
(Ministry) should establish and communicate 
clear criteria for what constitutes an eligible 
expense. 

In addition, the Ministry and agencies that 
administer SSAH funding should obtain suf- 
ficiently detailed invoices — and, where applic- 
able, receipts — to ensure that the amounts 
claimed are in fact eligible and reasonable 
before funds are disbursed. 

| MINISTRY RESPONSI 

In June 2011, the Ministry announced that it 
will be moving to a single direct funding pro- 
gram to make the system easier to navigate and 
more flexible for individuals and families. As of 
April 1, 2012, adults applying for direct funding 
support will apply to the Passport program. As 
outlined in the Ministry's response to Recom- 
mendation 4, the Ministry will be revising the 
Passport guidelines to specify more clearly the 
services and supports that can be purchased 
through this program and the reporting and 
accountability requirements. At the same time, 
the Ministry will also be reviewing its invoicing 
procedures to improve financial oversight. 

The SSAH program will continue to serve 
children and youth. There will be a review of 
the SSAH program guidelines to address the 
Auditor General's concerns. 



OTHER MATTERS 

Travel, Meal, and Hospitality Expenditures 

In the latter half of 2009, after questionable spend- 
ing practices at other public-sector organizations 
received significant public attention, the Ministry 
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of Finance announced that all agencies that receive 
government funding would have to comply with the 
government directive surrounding travel, meal, and 
hospitality expenses. The Ministry of Community 
and Social Services advised all its transfer-payment 
agencies to comply with the government directive, 
which, among other things: 

• states that expense claims must be properly 
documented and include detailed receipts; 

• outlines expenses that are not eligible 
for reimbursement, such as alcohol for 
employees; 

• defines under what conditions travel and 
accommodation expenses will be reimbursed; 
and 

• sets out acceptable hospitality costs. 

We found that the government's directive on 
travel, meal, and hospitality expenses had often not 
been adopted by agencies. 

We reviewed a sample of travel, meal, and hos- 
pitality claims of senior management. Most of these 
expenses were charged to agency credit cards. On an 
overall basis, we found that transfer-payment agen- 
cies often did not comply with the government's 
directive or with good business practices. We noted 
many instances where reimbursements for travel, 
meal, hospitality, and other expenses appeared 
excessive or otherwise inappropriate in our view. 
Our specific comments are detailed as follows. 

Travel 

We found several instances of travel to the United 
States where detailed invoices were not submitted 
to substantiate the expenses incurred. For example, 
invoices were not submitted for hotel accommoda- 
tions for the Hyatt Hotel in Phoenix, Arizona, where 
hotel charges to the agency credit card totalled 
$1,880. In another case, $1,300 was charged for 
accommodation at the Hilton Hotel in Seattle, with 
no details provided on the nature of the trip. In 
some cases where invoices were submitted, the cir- 
cumstances of the trips were not documented or jus- 
tified. For example, two people charged their agency 



credit cards a total of $3,587 for return flights to, 
and accommodations in, San Francisco. When ques- 
tioned, the agency explained that the purpose of the 
trip was for a "social enterprise conference." In addi- 
tion, we found that at some agencies, staff charged 
their credit cards for hotel accommodations in close 
proximity to their office headquarters, which is con- 
trary to the government directive. 

Meals and Hospitality 
Our review of a sample of meal and hospitality 
expenses charged to agency credit cards noted that 
many appeared excessive and/or were questionable 
in our view. They included: 

• $1,155 spent at a steakhouse, with neither 
the purpose nor the number and identities 
of those who attended stated, and with no 
detailed receipt submitted; 

• $1,090 spent at a steakhouse, with neither 
the purpose nor the number and identities of 
attendees stated, and with no detailed receipt 
submitted; 

• $747 for five cakes for a "top employers 
celebration"; 

• $570 spent on a "retirement lunch," with the 
number and identities of attendees not stated 
and no detailed receipt submitted; 

• $545 for catering for a "send-off reception" at 
which the number of guests was not recorded. 

Other types of questionable expenditures 
included: 

• gift cards totalling $800 purchased by an 
agency, with no record of who actually 
received the gift cards or why; 

• $327 spent on jewellery at Tiffany & Co. for a 
"retirement gift"; 

• annual lease and car insurance payments for 
a personal luxury vehicle totalling $11,000 
made with agency funding on behalf of the 
executive director. In addition, the executive 
director was reimbursed for all vehicle main- 
tenance and gas purchases. We also noted that 
the personal vehicle benefit obtained by the 
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executive director was not reported as a tax- 
able benefit on the individual's annual T4 slip. 
• fitness and pool memberships paid for by an 
agency in 2009 and 2010 worth $1,400 each 
year. The details of the memberships clearly 
identified that they were for two individuals — 
specifically, the executive director of the 
agency and the executive director's spouse. 
During the time of our audit, all agencies 
had to comply with the then government direc- 
tive on travel, meal, and hospitality expenses. 
However, the government's new Broader Public 
Sector Accountability Act, which came into effect in 
April 2011, stipulates that only agencies receiving 
$10 million or more per year in provincial funding 
must now comply with the new Broader Public Sec- 
tor Expenses Directive, which mirrors the govern- 
ment's 2009 Travel, Meal, and Hospitality Directive. 

The new directive notwithstanding, we believe 
that the principles in this directive provide sound 
guidance for all agencies to follow. 



RECOMMENDATION 9 



To help ensure that all agencies that are 
required to do so implement the government's 
new directive on travel, meal, and hospitality 
expenses, and that all other agencies follow 
the spirit of the directive, the Ministry of Com- 
munity and Social Services should reinforce the 
requirements to do so and consider having the 
agencies' board chairs annually attest to such 
compliance. 

MINISTRY RESPONSE 

The Ministry has strengthened its risk assess- 
ment process to include oversight of procure- 
ment activities and travel, meal, and hospitality 
expenses. The Ministry is now developing addi- 
tional measures and strategies to hold boards 
of directors accountable for the prudent use of 
program funds and compliance with the new 
Broader Public Sector Accountability Act, 2010, 
including board attestations for compliance 



and training for boards as recommended by the 
Auditor General. 

As noted by the Auditor General, effective 
April 2011, all ministry transfer-payment agen- 
cies that receive $10 million or more a year in 
provincial funding must now comply with the 
Broader Public Sector Expenses Directive, which 
mirrors the government's 2009 Travel, Meal, 
and Hospitality Directive. Agencies subject 
to the Act and Directive were notified of their 
obligations. 

Smaller agencies not subject to the Act were 
provided with the Broader Public Sector Direc- 
tives on Procurement and Expenses and were 
encouraged to voluntarily comply. 



SSAH Program Administration 

All nine of the Ministry's regional offices administer 
the SSAH program, which includes assessing clients 
for program eligibility and processing eligible reim- 
bursements. However, we noted some significant 
differences in the way the program is administered 
in some regions. 

Although all regional offices have similar staff- 
ing levels for administering the SSAH program, one 
office provided funding to six agencies to help it 
administer the program, at a cost of $2.1 million. As 
well, in five regions, a total of $3.2 million was paid 
to 33 agencies for helping SSAH clients fill out their 
application forms. The amount agencies received 
varied within regions and across the province. For 
example, some agencies received as little as $60 
on average per client served, while other agencies 
received as much as $1,500. The Ministry could not 
provide an explanation for the variances. 

In 2009, the Ministry established a working 
group to assess the appropriateness of the addi- 
tional administration expenditures being incurred. 
However, the Ministry had not taken action on any 
of the recommendations made by the group as of 
spring 2011. 
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RECOMMENDATION 10 



Given the similarities in overall staffing levels 
at the regional offices dedicated to the Special 
Services at Home (SSAH) program, the Ministry 
of Community and Social Services should assess 
the need for the additional administration costs 
being paid out to agencies and ensure that all 
costs incurred are reasonable and necessary 

INISTRYRESPONSI 

The Ministry agrees that program administra- 
tion costs should be reasonable and necessary. 
During summer and fall 2011, the Ministry was 
working to move toward a single direct funding 
program for adults with a developmental dis- 
ability. As part of this transition, the Ministry 
will be undertaking a review of SSAH program 
administration funding and guidelines. The 
Ministry will also review the administrative 
costs for the Passport program. 



Chapter 4 



Follow-up on 2009 
Value-for-money Audits 



It is our practice to make specific recommenda- 
tions in our value-for-money audit reports and 
ask ministries, agencies of the Crown, and organ- 
izations in the broader public sector to provide 
a written response to each recommendation, 
which we include when we publish these audit 
reports in Chapter 3 of our Annual Report. Two 
years after we publish the recommendations and 
related responses, we follow up on the status of 
actions taken by management with respect to our 
recommendations . 

Chapter 4 provides some background on the 
value-for-money audits reported on in Chapter 3 
of our 2009 Annual Report and describes the 
status of action that has been taken to address our 
recommendations since that time as reported by 
management. 

For a number of these audits, hearings were also 
held and reports issued by the Standing Committee 
on Public Accounts (Committee) . This year, for 
the first time, where a hearing was held, we have 
included a summary of the Committee's recommen- 
dations in the applicable section of this chapter. 



We believe that this additional reporting will help 
ensure that action is being taken by audited entities 
to address the issues that the Committee raised dur- 
ing the hearing and in its subsequent report to the 
Legislature. Chapter 6 describes the Committee's 
activities more fully. 

We are pleased to be able to report that for 
close to 90% of the recommendations we made in 
2009, progress is being made toward implementing 
our recommendations, with substantial progress 
reported for more than 40% of them. 

Our follow-up work consists primarily of inquir- 
ies and discussions with management and review of 
selected supporting documentation. In a few cases, 
the organization's internal auditors also assisted 
with this work. This is not an audit, and accordingly, 
we cannot provide a high level of assurance that the 
corrective actions described have been implemented 
effectively. The corrective actions taken or planned 
will be more fully examined and reported on in 
future audits and may impact our assessment of 
when future audits should be considered. 
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Ministry of Health and Long-Term Care 

Section 

Assistive Devices 
Program 



Follow-up on VFM Section 3.01, 2009 Annual Report 




Background 




The Ministry of Health and Long-Term Care (Min- 
istry) administers the Assistive Devices Program 
(Program), whose primary objective is to help 
provide personalized assistive devices to Ontario 
residents with long-term physical disabilities. In 
the 2010/11 fiscal year, Program expenditures 
were approximately $343 million ($347 million in 
2008/09). 

At the time of our 2009 audit, we found that the 
Ministry had improved its ability to monitor and 
enhance service delivery to clients since our previ- 
ous audit in 2001. However, we also concluded that 
the Program could be run more cost-effectively if 
the Ministry managed payments more economically 
and enforced eligibility and other policy require- 
ments more rigorously. In our 2009 Annual Report, 
some of our more significant observations were as 
follows: 

• A majority of people getting oxygen at home 
used oxygen concentrators that cost between 
$400 and $1,000 to buy and that last five to 
seven years, with the required periodic ser- 
vicing. However, the Ministry typically paid 
vendors about $23,000 over five years to buy 
and service each device without analyzing 
whether this cost was reasonable. 



• The Ministry set 33% as a reasonable rate of 
return for vendors of assistive devices. How- 
ever, we found that average vendor mark-ups 
for mobility aids, respiratory devices, and com- 
puter systems were 84%, 117%, and 128%, 
respectively. In setting prices, the Ministry did 
not consider significant price decreases arising 
from technological advances or the volume 
discounts available to some vendors. 

• Vendors were receiving even greater rates 
of return from computer components such 
as monitors, printers, and scanners. For 
example, one monitor that often cost vendors 
only about $250 had a Program-approved 
price of $1,332 — a potential return of 400%. 
We also found that vendor price quotes for the 
same computer system varied significantly, 
from $1,300 to $4,400. 

• The Ministry was not consistently monitoring 
scooter claims to identify unusual patterns; 
nor was it taking appropriate action to prevent 
potential abuses. We noted that scooter claims 
of some vendors increased by more than 
800% over the last three years. 

• In our sample, one-third of the assessments 
that should have been conducted by vendors 
to confirm clients' continued eligibility for 
home oxygen either had not been done or 
showed that the clients no longer qualified. 
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Yet the Ministry was not made aware of this 
and continued to pay for their home oxygen. 

• Claims for Frequency Modulated (FM) 
systems, a more expensive type of hearing 
aid, increased dramatically among seniors, 
from $250,000 in 2004/05 to $4.8 million in 
2008/09. However, some clients indicated that 
they did not really need or use the FM system. 

• We found that some vendors had more than 
90% of their claims authorized by only one 
or two health-care professionals. One such 
vendor had since 2000 claimed more than 
$10 million for hearing aids. Some author- 
izes regularly referred clients to the same 
vendors, even when there were others much 
closer to the client's residence. The Ministry 
had known about some of these cases for sev- 
eral years but took no action. 

• Ontario did not recycle used manual wheel- 
chairs. Other provinces, such as Alberta and 
Quebec, achieved cost savings of $4 million to 
$5 million per year and protected the environ- 
ment by recycling manual wheelchairs. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 

STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 

In March 2010, the Standing Committee on Public 
Accounts (Committee) held hearings on our audit 
and expressed concerns about the Ministry's 
progress in addressing many of the issues we had 
identified. In August 2010, the Chair of the Com- 
mittee wrote to the Deputy Minister, questioning 
the Program's business model and listing the Com- 
mittee's specific concerns. In November 2010, the 
Ministry was called back for a follow-up hearing, 
and the following May, the Committee issued a 
report. One of the Committee's recommendations 
asked our Office to follow up on its five areas of 
concern as follows: 



• Volume discounts — to examine the Ministry's 
progress in capturing volume discounts while 
still maintaining equitable access across the 
province. 

• Inter-jurisdictional price comparisons — to 
assess whether the Ministry is comparing the 
prices it pays in major device categories to the 
prices being paid by other jurisdictions. 

• IT system — to determine whether the Ministry 
is meeting its deadlines for implementing 

its new system and whether the new system 
is helping to reduce processing time for 
applications. 

• Claims backlog — to determine whether the 
Program has made progress in eliminating its 
claims backlog and achieved its processing 
time target of six to eight weeks. 

• Increased auditing and evaluation of vend- 
ors — to assess whether the Ministry has imple- 
mented stronger procedures to prevent and 
detect potential program abuses. 

Accordingly, as part of our follow-up work, 
we also reviewed the status of ministry actions to 
address the Committee's concerns. 



Status of Actions Taken on 
Recommendations 



The Ministry implemented a Modernization Project 
(Project) from fall 2010 to summer 2011 to redesign 
the Assistive Devices Program (Program) that 
addressed most of our 2009 recommendations. 
The Project also incorporated a new IT system to 
help improve program management and delivery. 
The Project included such initiatives as changing 
Program-approved prices for computers and mobil- 
ity devices, having an external consultant undertake 
a pricing and funding model review, and launching 
a manual-wheelchair-recycling pilot project. 

According to information provided to us by the 
Ministry, significant progress has recently been 
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made in addressing most of our recommendations 
and those of the Standing Committee on Public 
Accounts (Committee). The Ministry advised us 
that it will take additional time to fully address 
certain of the recommendations. The status of the 
action taken on each recommendation at the time 
of our follow-up was as follows. 

PRICING 

Pricing of Home Oxygen 

Recommendation 1 

To ensure that prices for home oxygen are competitive, 
the Ministry of Health and Long-Term Care should 
perform a more rigorous analysis of the costs of deliv- 
ering home oxygen under each method before nego- 
tiating the new rate for home oxygen. This analysis 
should consider the oxygen prices other provinces are 
paying to ensure that Ontario is getting good value, 
especially given the economies of scale that should 
result from being the largest province. 

The Ministry should seek clarification from the 
Management Board of Cabinet with respect to the 
approval not to tender for home oxygen provided 
that "total expenditures for the program should not 
exceed $54.6 million annually." Specifically, it should 
confirm whether the maximum can be exceeded due to 
an increase in utilization provided the increase can be 
funded internally within the Ministry and approved 
through a Treasury Board Order. 

Status 

The Program implemented a Vendor of Record 
(VOR) system on April 1, 2010, with 69 home oxy- 
gen vendors on the list. The VOR system requires 
vendors to report information on the type of 
oxygen systems they supply to clients. The Ministry 
indicated that since April 1, 2011, the Program has 
tracked this information to help it understand the 
cost of delivering home oxygen. As of June 2011, 
about 70% of 13,600 clients were using stationary 
concentrators with cylinders, but the Program had 
yet to collect information from about 20,400 other 
clients. As a result of the VOR system, the Program 



has saved about $2 million annually by more accur- 
ately capturing the start and end dates of home 
oxygen services to ensure that funding is initiated 
at the start of oxygen therapy and terminated when 
therapy stops. 

Because the VOR system is relatively new, no 
changes in the service delivery model are currently 
envisioned. However, the Ministry indicated that 
it has begun a statistical review, to be concluded 
by April 2013, to provide baseline data for pricing 
updates. 

In spring 2008, the Program undertook a review 
of home oxygen programs in other jurisdictions, 
paying particular attention to programs in Saskatch- 
ewan and Alberta. A second jurisdictional review 
was conducted in September 2010 to validate equip- 
ment and maintenance costs, as well as replacement 
periods. The review noted that the $1,172 cost 
in Ontario for 90-day funding is very close to the 
$1,208 in Saskatchewan and $1,155 in Alberta. The 
Program will conduct pricing reviews and consider 
updating prices accordingly in April 2013. 

The Ministry also sought and received approval 
from the Treasury Board Office to allow expendi- 
tures for the home oxygen program to exceed 
$56.4 million annually, provided the increase can 
be funded internally by the Ministry and approved 
through a Treasury Board Order. 

Pricing of Other Devices 

Recommendation 2 

To ensure that the cost of equipment paid for by the 
Ministry and its clients is competitively priced, the 
Ministry of Health and Long-Term Care should: 

• conduct regular pricing reviews for each device 
category and update Program-approved prices 
accordingly; and 

• take volume discounts and technological advan- 
ces into consideration when updating Program- 
approved prices. 
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Status 

The Ministry reviewed and updated Program- 
approved prices in several categories, including 
computer systems, mobility devices, orthotics, and 
ocular prostheses. Specifically, according to the 
Ministry: 

• Program-approved prices for computer and 
related equipment in the Communication Aids 
and Visual Aids categories were reduced by 
about 60% effective January 1, 2011, to better 
reflect current market prices. The price cuts 
are expected to save the Program $2.2 million 
annually. 

• Effective April 1, 2011, Program-approved 
prices for mobility devices were revised, 
which should generate savings of about 
$1.2 million a year. 

• Program-approved prices for orthotics were 
increased effective April 1, 2011, based on 
information about material costs supplied by 
the Ontario Association of Prosthetists and 
Orthotists. The increases are expected to cost 
the Program about $600,000 more a year. 

• Effective April 1, 2011, Program-approved 
prices for ocular prostheses were increased 
after discussions with certified ocularists and 
with staff in other Canadian jurisdictions. The 
increases are expected to cost the Program 
about $500,000 more annually. 

Apart from the above pricing changes, Program 
staff also conducted jurisdictional reviews on insu- 
lin pumps and Continuous Positive Airway Pressure 
(CPAP) machines and concluded that no price 
changes were needed for these device categories. 

The Program is currently conducting similar 
reviews of other high-volume, high-cost devices, 
and the Ministry expects to complete a review of all 
device categories by summer 2012. The review will 
include price comparisons with Alberta, Quebec, 
and Saskatchewan. 

In addition, as part of the Modernization Pro- 
ject, the Program has engaged external experts to 
assist with a comprehensive funding model and 
pricing review of all device categories. The review 



aims to identify ways to capture volume discounts, 
particularly for high-volume, high-cost device cat- 
egories, which offer the biggest potential savings. 
The Ministry also advised us that the Program will 
in future conduct ongoing pricing reviews and com- 
parisons with other jurisdictions. 

VERIFICATION AND REVIEW PROCESS 

Monitoring of Claims 

Home Oxygen Claims 
Recommendation 3 

To ensure that funding for home oxygen is provided 
only to individuals who require it for medical reasons, 
the Ministry of Health and Long-Term Care should: 

• assess whether more stringent vendor oversight 
is required to ensure that the required periodic 
assessment tests are being appropriately con- 
ducted and reported, or, alternatively, consider 
the practicality of having independent respira- 
tory therapists perform eligibility assessments, 
rather than vendors' staff ; and 

• establish procedures and assign clear respon- 
sibility for discontinuing home oxygen supply to 
clients who no longer meet the medical eligibility 
criteria. 

Status 

The Ministry advised us that since 2008/09, the 
Program had recovered about $485,000 from home 
oxygen vendors who continued to bill the Ministry 
after a client had died. In October 2010, the Min- 
istry found more cases of overpayment, including 
clients who had died in long-term-care homes but 
who were still being funded for oxygen therapy, and 
the Program recovered about $106,000 dating back 
to 2006. In the first half of 2011/12, the Ministry 
identified an additional $132,000 for recovery, and 
in June 2011, a new information system — Assistive 
Devices Application Management — was launched 
that the Ministry expected would help detect simi- 
lar claim anomalies in the future. 

According to the Ministry, the new Vendor of 
Record (VOR) system for home oxygen services 
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included new mandatory requirements to ensure 
that funding would be provided only to eligible 
individuals. Specifically: 

• In addition to the initial, 90-day, and one-year 
physical assessments, subsequent annual re- 
assessments will be required to confirm the 
client's ongoing need for oxygen therapy. 

• The VOR system states that discontinuation 
of oxygen therapy is the responsibility of the 
client's physician and is based on the phys- 
ician's assessment of a client's medical needs. 

Mobility Aids-Scooter Claims; Hearing Aids-FM 
System Claims; Ostomy Supply Claims; and 
Insulin Pump and Supply Claims 
Recommendation 4 

To ensure that Assistive Devices Program funding for 
devices and supplies is provided only to individuals 
who are eligible for it, the Ministry of Health and 
Long-Term Care should: 

• identify and investigate abnormal claim pat- 
terns through regular reviews; 

• take action to deter authorizers or vendors who 
are suspected of abusing or misusing program 
funding, including suspending their registration 
with the Program and bringing the matter to the 
attention of the appropriate regulatory college 
or professional association where professional 
misconduct is suspected. 

Status 

The Ministry informed us that it has taken the fol- 
lowing actions to ensure that Program funding for 
devices is provided only to eligible individuals: 

• It launched a new information system, called 
Assistive Devices Application Management, 
in June 2011 to help detect abnormal claim 
patterns by examining claim patterns, 
authorizer-vendor links, and patterns within 
device categories. 

• The Program has developed a Claims Verifica- 
tion and Review Policy, which requires regular 
reviews of claims and claim patterns for all 
device types. 



• The Program worked with the Ministry's 
Accounting Policy and Financial Reporting 
Branch (Branch) in order to identify and 
investigate unusual claim patterns. The 
Branch indicated that it continues to review 
claims data. It also targets areas of high risk, 
and verifies and tests claims samples from all 
device categories. 

The Ministry also indicated that the Program 
made changes to the way it handles claims for 
Frequency Modulated (FM) hearing devices after 
learning of the significant increase in such claims 
since 2006/07, as follows: 

• In January 2009, the Program and the Min- 
istry's Fraud Awareness and Management Unit 
developed the FM System Review Work Plan 

to prevent abuses by, among other things, 
making Claims Assessors and Program Co- 
ordinators responsible for monitoring and 
reviewing claims. 

• Since 2009/10, the Program has recovered 
$243,000 from eight vendors, and has identi- 
fied an additional $4.4 million from 40 vend- 
ors for potential recovery. There has also been 
a dramatic decrease in claims, from more than 
5,000 in 2008/09 to about 1,000 in 2009/10. 

• The eligibility criteria for FM systems were 
updated and stated clearly on the application 
form and in the administration manual for 
hearing devices. 

Post-payment Review Process and Fraud 
Investigation 

Recommendation 5 

To more effectively identify abuses, recover overpay- 
ments, and deter misconduct, the Ministry of Health 
and Long-Term Care should: 

• expand its efforts and resources to better mon- 
itor vendors' and authorizers' compliance with 
program policies and procedures; 

• take timely corrective action to terminate agree- 
ments with vendors and authorizers who have 
clearly violated program policies; 
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• work with the Ministry's Accounting Policy and 
Financial Reporting Branch to elevate staff risk- 
awareness and risk-assessment skills; and 

• where there is clear evidence of potential mis- 
conduct, report its concerns to the appropriate 
regulatory associations or colleges, which are 
responsible for ensuring the public is protected. 

Status 

The Ministry indicated that since July 2006, the 
Program had referred eight cases to the Ontario 
Provincial Police for investigation of suspected 
fraud. Three cases were closed without charges, 
one was resolved with $560,000 recovered, one 
resulted in court-ordered restitution, and three 
were still under investigation. The Ministry's 
Accounting Policy and Financial Reporting 
Branch (Branch) said that from November 2009 
to July 2011 it had recovered $1.8 million in 
overpayments. 

The Ministry took action where there was 
evidence of potential misconduct and violation of 
Program policies by vendors and authorizers: see 
the status of Recommendations 6 and 9. 

The Branch provided risk management and 
fraud awareness training sessions in Septem- 
ber 2010 to Program staff, and additional risk 
management training for new staff was scheduled 
in July 2011. The Ministry informed us that other 
training will be offered in future on an ongoing 
basis to provide Program staff with specific learning 
and training opportunities to improve verification 
and claims review. 

CONFLICT OF INTEREST 

Recommendation 6 

To deter potential conflict of interest as well as the 
misuse and abuse of program funding, the Ministry of 
Health and Long-Term Care should: 

• more closely monitor vendor billing patterns 
and, particularly when claims have increased 
dramatically, consider investigating the various 



parties for evidence of inappropriate author- 
izing or billing practices; 

• terminate agreements with vendors and author- 
izers who breach the Program's conflict of inter- 
est policies; and 

• inform the appropriate regulatory college or 
professional association of any health-care pro- 
fessionals whose behaviour or practices put the 
public at risk of harm. 

Status 

The Program has strengthened its Conflict of Inter- 
est Policy and Procedures for Managing Breach of 
ADP Vendor and Authorizer Agreements, which 
specifies the process leading up to suspension 
and/or termination of contracts with vendors 
and authorizers after the Program has identified 
a breach of contract. The new procedures were 
posted on the Program's website in November 2010, 
and notices were mailed to all registered vendors 
and authorizers. The Program has also designated 
staff to respond to inquiries about conflict of 
interest. 

In June 2011, a new information technology 
system, Assistive Devices Application Manage- 
ment, was launched to help detect abnormal claim 
patterns. The system generates regular reports 
on claim patterns, authorizer-vendor links, and 
patterns within device categories. According to 
the Deputy Minister's presentation to the Standing 
Committee on Public Accounts in November 2010, 
the full benefits of the new system will likely 
become apparent only in early 2012. At that time, 
the Program will be in a better position to conduct a 
quantitative benefit analysis. 

The Ministry indicated that although the 
Program has not referred any case of professional 
misconduct to any regulatory college since 2008, 
matters related to fraudulent billings by vendors 
have been referred to the Ontario Provincial Police. 
The Program has conducted regular reviews to 
ensure that contractual commitments with author- 
izers are enforced. As a result of the reviews in 
February and May 2011, the Program terminated 
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authorizer agreements with seven physiotherapists, 
35 occupational therapists, and 21 audiologists 
who were not in good standing with their regula- 
tory colleges. 

RECYCLING AND REFURBISHING 
INITIATIVES 

Recommendation 7 

To achieve cost savings and protect the environment, 
the Ministry of Health and Long-Term Care should 
consider the feasibility of implementing a strategy to 
recycle and refurbish used manual wheelchairs based 
on the experience of other jurisdictions that have suc- 
cessfully adopted such a strategy. 

Status 

The Program reviewed recycling programs in other 
jurisdictions and determined that their adminis- 
trative and service delivery models vary widely. 
According to the Ministry, no single recycling model 
that is in use elsewhere would completely meet 
Ontario's needs, given the broad range of wheel- 
chairs and seating devices funded by the Program 
in this province. 

In order to determine the potential market for 
recycled wheelchairs in Ontario, the Ministry has 
entered into an agreement with the Canadian Red 
Cross to run the Manual Wheelchair Recycling Pilot 
Project, launched on June 30, 2011, to assess the 
availability of certain types of manual wheelchairs 
for recycling. Although these wheelchairs would 
be collected across the province, they would be dis- 
tributed only in the Hamilton region. The Program 
will assess the pilot project's effectiveness after one 
year. 

RECOVERY OF OVERPAYMENTS 

Recommendation 8 

To ensure that Assistive Devices Program grants are 
administered economically, the Ministry of Health 
and Long-Term Care should recover overpayments on 



a timelier basis and expedite the recovery of overpay- 
ments made since 2005. 

To ensure that funding for devices is not duplicated 
at taxpayers' expense, the Ministry of Health and 
Long-Term Care should re-institute an information- 
exchange agreement with the Workplace Safety and 
Insurance Board and initiate an agreement with the 
Department of Veterans' Affairs as has been recom- 
mended by the Ministry's Fraud Programs Branch. 

Status 

The Ministry indicated that since 2008/09, the Pro- 
gram has recovered about $334,000 from the Work- 
place Safety and Insurance Board (WSIB) related 
to duplicate funding for hearing aids. The Ministry 
also signed an information-sharing agreement with 
the WSIB in summer 2011. 

The Program continues to recover funds from 
the WSIB when it has been determined that a client 
is eligible for WSIB funding for a hearing aid due 
to a workplace-related injury, and it has been in 
discussions with Veterans Affairs Canada (VAC), 
which is willing to develop an information-sharing 
agreement. The Program has implemented new 
forms that require a client to indicate whether he or 
she is eligible for WSIB or VAC benefits. If the client 
indicates eligibility for either, the Program will 
reject the claim and refer it accordingly. All new 
forms also seek client consent to allow the Program 
to share information with the WSIB and VAC. 

REGISTRATION 0FAUTH0RIZERS 

Recommendation 9 

To lower the risk of assistive devices being approved 
for funding by authorizers who are not properly regis- 
tered with the Program, the Ministry of Health and 
Long-Term Care should: 

• generate links with the professional colleges 
to enable ongoing monitoring of authorizers' 
status; and 

• follow up on those authorizers who do not sub- 
mit the required Information Update Forms. 



Assistive Devices Program 



Status 

The Ministry indicated that it took the follow- 
ing measures to reduce the risk of devices being 
approved for funding by authorizers improperly 
registered with the Program: 

• The Program met with regulatory colleges 
and signed information-sharing agreements 
representing more than 99% of the Program- 
registered authorizers. The Program will 
manually verify the status of those authorizers 
not covered by an information-sharing agree- 
ment by checking the respective regulatory 
college website. 

• The Program regularly reviewed compliance 
with authorizer agreements. Previously 
the Program used to send out Authorizer 
Confirmation Notices every three years, 
requesting that authorizers confirm their 
status as members in good standing of their 
regulatory colleges. Such confirmation will 
now be required at least once every year. 
Failure to meet this requirement may lead 
to suspension or termination of authorizers' 
registration with the Program. 

• The Program posted a new Authorizers' Roles 
and Responsibilities document on its website 
in January 2011. The document collects in one 
place all of the information contained in pro- 
gram manuals and agreements that pertain to 
authorizers' roles and responsibilities. 



on each recommendation at the time of our follow- 
up was as follows. 

VOLUME DISCOUNTS 

Committee Concern 1 

The Auditor should examine what progress the Min- 
istry has made in capturing volume discounts while 
still addressing issues related to providing equitable 
access to the Program across Ontario. If the Ministry 
is not yet capturing these discounts, it should explain 
to the Auditor its plan for doing so, including a 
timeline. 

Status 

This was addressed in the Status section of our 
Recommendation 2. 

INTER JURISDICTIONAL PRICE 
COMPARISONS 

Committee Concern 2 

The Auditor should assess whether the Ministry is 
conducting inter-jurisdictional price comparisons in 
major device categories besides home oxygen pricing. 
The Ministry should, for example, provide documen- 
tation of price comparisons made for various device 
groups. 

Status 

This was addressed in the Status sections of our 
Recommendations 1 and 2. 




Information provided to us by the Ministry indi- 
cated that significant progress had been made 
in addressing most of the concerns raised by the 
Committee in November 2010, but the Ministry 
acknowledged that it will take additional time to 
fully address all of them. The status of action taken 



IT SYSTEM 

Committee Concern 3 

The Auditor should determine whether the Ministry 
is meeting its deadlines for implementation of its 
new IT system and whether the new system is help- 
ing to reduce the amount of time required to process 
applications. 

Status 

This was addressed in the Status sections of our 
Recommendations 3, 4, and 6. The Ministry met 
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its deadline for implementation of a new IT system 
in June 2011 with Assistive Devices Application 
Management. According to the Ministry, the new 
system is expected to reduce data entry time and 
errors with category-specific, consistent, and easy- 
to-use forms. It should also help reduce assessment 
times. Given the newness of the system, actual 
performance could not be assessed at the time of 
our follow-up. 

CLAIMS BACKLOG 

Committee Concern 4 

The Auditor should determine whether the Program 
met its January 2011 deadline to begin making 
progress on eliminating its claims backlog and also 
determine the Program's progress in achieving its 
targeted six to eight week processing timeframe. 

Status 

A claim-processing backlog developed in 2010 
following a 62% increase in demand, aggravated 
by process issues and a staff shortage. In Sep- 
tember 2010, the Program began to track claim- 
processing times and found that, since March 2011, 
the Program has been processing claims for major 



device categories within the approved service stan- 
dard of six to eight weeks. 

According to data provided by the Program, 
average claim-processing times for all major and 
high-volume device categories have been reduced 
significantly, from over 10 weeks during summer 
2010 to five weeks or less in May 2011. For example, 
average claim-processing time for home oxygen 
dropped from 23 weeks to three; for mobility aids, 
from 20 weeks to two; for respiratory devices, from 
15 weeks to four; and for hearing aids, from 12 
weeks to five. 

INCREASED AUDITING AND EVALUATION 
0FVEND0RS 

Committee Concern 5 

The Auditor should assess whether the Ministry has 
implemented strengthened procedures to prevent and 
detect potential program abuses through increased 
auditing and monitoring of vendors and vendor bill- 
ing patterns. 

Status 

This was addressed in the Status sections of our 
Recommendations 4, 5, and 6. 



Chapter 4 



Ministry of Transportation 

Section 

14.02 Bridge Inspection 

and Maintenance 

Follow-up on VFM Section 3.02, 2009 Annual Report 



Background 



Ontario has about 14,800 bridges. Approximately 
2,800 of these are located within the provincial 
highway system and are the responsibility of the 
Ministry of Transportation (Ministry) . The remain- 
ing 12,000 are located in municipalities and are 
their responsibility. 

Responsibility for the safety and maintenance of 
provincial bridges is set out in the Public Transpor- 
tation and Highway Improvement Act (Act). The Act 
requires that all provincial and municipal bridges 
be inspected every two years under the direction of 
a professional engineer using the Ministry's Ontario 
Structure Inspection Manual (Inspection Manual) . 
The Inspection Manual requires these biennial 
inspections to be a "close-up" visual assessment of 
each element of a bridge to identify any material 
defects, performance deficiencies, or maintenance 
and rehabilitation needs. 

PROVINCIAL BRIDGES 

In our 2009 Annual Report we noted that the Min- 
istry had established comprehensive standards for 
bridge inspection in the Inspection Manual, and if 
the standards are followed, the required inspection 
procedures effectively enable structural deficiencies 



to be identified. The Ministry was also conducting 
bridge inspections on a biennial basis as required. 

However, we noted a number of areas where 
improvements to the Ministry's inspection and 
maintenance processes would help minimize poten- 
tial safety risks — such as those caused by falling 
concrete or by parts of a bridge structure failing 
to perform their intended function of providing 
adequate protection to the vehicles travelling on 
or underneath the structure — and would ensure 
that bridges for which the province is responsible 
remain safe. Our observations were as follows: 

• According to the Ministry's assessment, more 
than 180 or 7% of provincial bridges were in 
poor condition, defined as requiring repair 
or rehabilitation work within one year of the 
bridge inspection. We found that, despite 
their being in most need of repair or rehabili- 
tation, over one-third of these bridges were 
not included in the Ministry's capital work 
plan for the upcoming year. 

• The Ministry had not ensured that informa- 
tion on critical elements within each bridge 
was accurate and that all elements were 
accounted for. The state of these elements is 
the key to determining a bridge's overall con- 
dition and estimating any needed rehabilita- 
tion costs. In addition, the Ministry's database 
of bridge inventory — the Bridge Management 
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System (BMS) — did not have information on 
the rehabilitation history for almost one-third 
of the bridges that were 40 years or older. 
The Inspection Manual requires a detailed 
visual "close-up" inspection of each bridge 
element. Normally, this requires the closure 
of lanes and road shoulders to traffic. For 
example, without closing a lane, close-up 
inspection of the critical elements of certain 
bridges on Highway 401 in the Greater 
Toronto Area would not be possible, yet there 
had been no such lane closures for the previ- 
ous three years at the time of our 2009 audit. 
We found several weaknesses regarding the 
process for ongoing oversight of inspections. 
For example: 

• The Inspection Manual stipulates that an 
inspector needs to spend at least two to 
three hours at a typical bridge site. How- 
ever, inspectors were often conducting five 
or more inspections a day. For example, in 
the rounds of inspections between 2006 
and 2008, we noted that 10 or more bridges 
were inspected by a single inspector in one 
day on 36 separate occasions. 

• A significant change in the rating of a 
bridge's condition between inspections 
requires explanation and, potentially, a 
re-inspection. We noted that the latest 
inspection results at the time of our 2009 
audit showed an improvement in the over- 
all condition rating of over 300 bridges, 
even though little or no rehabilitation work 
had been done on these bridges since the 
previous inspection. In other instances, the 
overall rating did not change at all between 
inspections, and reports from the previous 
inspections were carried forward without 
any changes. Although in many cases there 
were photographs on file to indicate that 
an inspection had been done, when no 
changes whatsoever in the condition of the 
bridge had been noted since the last inspec- 
tion, the adequacy of at least some of these 



inspections should have been followed up 
on, especially on older bridges, because a 
bridge's elements typically deteriorate over 
time. 

• We noted that regions tended not to complete 
many of the maintenance recommendations 
resulting from biennial bridge inspections. In 
two of the three regions that we visited, only 
about one-third of the recommended main- 
tenance work was actually completed, and the 
third region did not track this work at all. 
With respect to the procurement of major pro- 
jects for bridge design and construction, we noted 
that the Ministry generally followed a competitive 
selection process. However, in many of the con- 
tracts for design services and construction oversight 
consulting that we examined, there were changes 
to the scope of work that resulted in a final price of 
at least 50% more than the original contract price. 

MUNICIPAL BRIDGES 

To ensure the safety of municipal bridges, muni- 
cipalities are also required to perform biennial 
inspections in accordance with the Inspection 
Manual. At the time of our audit, we noted that 
there was no legislation that requires or even 
enables the Ministry of Transportation or any 
other provincial ministry to oversee municipalities' 
compliance with this requirement. There was also 
no central database on the number of municipal 
bridges and their overall condition. 

Our survey of municipalities indicated that the 
average age of municipal bridges was generally 
higher than that of provincial bridges. However, it 
was not possible to get an accurate picture of the 
overall condition of municipal bridges or to make 
accurate comparisons between municipal and 
provincial bridges, because municipalities use many 
different systems to classify and determine the con- 
dition of their bridges. Nevertheless, the majority of 
municipalities (85%) that responded to our survey 
indicated that they had a backlog of rehabilitation 
work. Large and growing communities generally 
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did not have significant backlogs because their 
infrastructure was newer, in contrast to munici- 
palities with a large number of bridges relative to 
their population and revenue base, which had more 
difficulty funding bridge rehabilitation. 

The province had provided municipalities with 
one-time funding for municipal capital projects. 
However, funding decisions were often made on the 
basis of population and the network of roads rather 
than specific needs relating to bridges. As well, the 
funds were paid close to the end of the province's 
fiscal year, and many municipalities were not able 
to properly plan and spend the money. For instance, 
a significant portion of the funds provided in 2008 
remained unspent one year later. Municipalities 
told us that better asset-management practices 
supported by more sustainable provincial funding 
were needed to ensure safety and maximize the 
lifespan of their bridges. At the time of our audit, a 
provincial-municipal working group was examin- 
ing these issues. 

STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this audit in March 2010. In Novem- 
ber 2010, the committee tabled a report in the 
Legislature resulting from this hearing. The report 
contained nine recommendations and requested 
the Ministry to report back to the Committee with 
respect to the following: 

• changes being made to ministry policies and 
practices to identify and differentiate between 
bridge deficiencies that pose a safety risk and 
those that indicate a loss in economic value, 
and whether all provincial bridges rated fair to 
poor had now been included in the Ministry's 
five-year capital plans; 

• how the Ministry would provide more guid- 
ance on the practice of lane and shoulder 
closures in its Inspection Manual to allow both 
its staff and contract inspectors to perform 
consistent and effective bridge inspections; 



• whether the Ministry had monitored the 
effectiveness of its enhanced oversight initia- 
tives and inspection training for its staff and 
external engineering consultants, and the 
results of its monitoring, including whether 
significant increases or decreases in a struc- 
ture's Bridge Condition Index from one inspec- 
tion to the next were being followed up on; 

• steps the Ministry had taken to better track 
and explain any incomplete work relative to 
scheduled maintenance for the year; 

• steps the Ministry had taken to integrate mis- 
sing information and to correct inaccuracies 
and discrepancies in its inventory of provincial 
bridges and their elements; 

• the Ministry's conclusions stemming from its 
interim evaluation of its project to track and 
monitor the variance between estimated and 
actual design costs, and the results to date of 
its "smart sourcing" initiative; 

• the status of the Roads and Bridges Review 
Study being conducted jointly by provincial 
and municipal representatives (the Commit- 
tee also requested the Ministry to direct the 
review process to include possible options 
for the creation of a central oversight body to 
monitor biennial bridge inspection and main- 
tenance activity at the municipal level) ; 

• the Ministry's views on the merits of having a 
uniform bridge information and management 
system among municipalities, along with a 
report on the feasibility of making the Min- 
istry's BMS available to municipalities for the 
purpose of providing better information on 
bridge inspection and maintenance processes 
at the local level; and 

• a proposal that could enable the allocation of 
infrastructure funds from the province to pri- 
ority municipal bridge improvement or repair 
projects where safety is the key criterion. 

The Ministry formally responded to the Commit- 
tee in February 2011. A number of the issues raised 
by the Committee were similar to our observations. 
Where the Committee's recommendations are 
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similar to ours, this follow-up includes the recent 
actions reported by the Ministry to address the con- 
cerns raised by both the Committee and our 2009 
audit. 





Status of Actions Taken on 
Recommendations 



The Ministry provided us with information in 
spring 2011 on the current status of the actions 
taken on our recommendations. According to this 
information, significant progress has been made in 
addressing many of the recommendations we made 
in our 2009 Annual Report with regard to provincial 
bridges, although some will require more time to 
address fully. Our concerns with regard to muni- 
cipal bridges have been only partially addressed, 
since data collection and a provincial-municipal 
review were still under way at the time of this 
follow-up. The status of action taken on each of our 
recommendations at the time of our follow-up was 
as follows. 

SAFETY OF PROVINCIAL BRIDGES 

Recommendation 1 

To ensure that appropriate and timely action is taken 
on bridges requiring repair and rehabilitation work, 
the Ministry of Transportation should: 

• strengthen its risk-assessment and priority- 
setting process, with particular consideration 
given to bridges identified as being in poor 
condition, so that any urgently required work is 
given first priority; and 

• ensure that government decision-makers receive 
the information they require to adequately 
assess both safety and economic risks in order 
to prioritize the capital needs of Ontario's aging 
provincial bridges. 



Status 

At the time of our follow-up, the Ministry indicated 
that it had strengthened its policies and procedures 
to identify and record safety-related defects by 
requiring that: 

• bridge inspectors identify all urgent items in 
the comments section of the bridge inspection 
form and notify the appropriate ministry rep- 
resentative; and 

• the nature of the work completed or other 
action performed is also recorded in the com- 
ments section of the bridge inspection form to 
provide a permanent record of the work done. 

In addition, mandatory bridge inspection work- 
shops held subsequent to our audit emphasized the 
process for identifying safety-related deficiencies. 

The Ministry also indicated that it now requires 
the completion of a justification form that explains 
why any bridge with a Bridge Condition Index 
(BCI) of less than 60 is not on the five-year capital 
construction program and what measures are being 
taken to ensure the safety of the bridge. 

The Ministry indicated that, to ensure that gov- 
ernment decision-makers receive the information 
they need to prioritize the capital needs of Ontario's 
aging provincial bridges, it completed in September 
2011 multi-year regional investment plans that list 
the needs and corresponding investments required 
for bridges and pavements over a 25-year period. 
The plans include information on bridge structure 
needs, construction costs, the recommended year 
for the improvements, as well as the projected out- 
come of the investments. 

BRIDGE INVENTORY 

Recommendation 2 

To better ensure that the results of bridge inspections 
are accurately recorded and to better prioritize and 
estimate the cost of bridge repair and rehabilitation, 
the Ministry of Transportation should: 

• more closely monitor inspectors' compliance 
with the Bridge Inspection Manual so that 
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critical bridge information is accurately 
updated; and 

• act on findings from its quality-assurance review 
and ensure the completeness and accuracy of 
information kept in the Ontario Bridge Manage- 
ment System. 

Status 

In fall 2009, the Ministry issued a policy memo that 
requires inspectors to review the accuracy of the 
information on bridge inventory and the individual 
elements contained in each bridge as part of the 
inspection. Ministry engineers are required to 
conduct spot checks to ensure compliance with this 
requirement. 

The Ministry also initiated a multi-phased pro- 
ject to ensure the accuracy of bridge information in 
the BMS. The project includes: 

• identifying large differences between a 
bridge's deck area as recorded in its design 
drawings and its BMS data; 

• ensuring that the BMS contains sufficiently 
detailed information for all bridges (the BMS 
"key aspects field") ; 

• ensuring that the last rehabilitation date has 
been entered in the system, where applicable; 

• reviewing bridge drawings to ensure that the 
inventory data in the BMS are accurate for 
those bridge elements that have the largest 
impact on the BCI; and 

• confirming during field inspections the 
information on the elements of each bridge 
contained in its design drawings. 

The Ministry indicated that its Bridge Office will 
review the inventory of all bridges and the data on 
the elements contained in each, after this informa- 
tion has been corrected at the regional level. This 
work is scheduled to be completed by December 
2012. The Bridge Office's field audits, which involve 
re-inspections of 50 bridges annually, will now 
include a review of inventory and bridge element 
data. 

Starting in 2011, the Ministry also increased the 
frequency of its quality assurance inspections from 



a biennial to an annual basis. The inspections now 
include a review of recommendations from previ- 
ous quality assurance inspections and a report on 
the status of those recommendations. 

GAINING ACCESS TO BRIDGES FOR 
INSPECTION 

Recommendation 3 

To ensure that inspections are carried out in accord- 
ance with legislation, the Ministry of Transportation 
should: 

• arrange for the closure of lanes and shoulders 
whenever these are required to ensure that an 
adequate bridge inspection can be carried out; 

• if closure of lanes and shoulders is not always 
possible for every bridge inspection, consider a 
risk-based approach that takes into considera- 
tion factors such as the age of the bridge and 
the feasibility of rotating inspections. Off-peak 
closures such as at night or on weekends also 
warrant more consideration to facilitate bridge 
inspection; and 

• consider specifying lane and shoulder closures 
when tenders are issued for inspections to be 
done by external consultants. 

Status 

At the time of our follow-up, the Ministry informed 
us that the policy memo issued in 2009 makes it 
mandatory for bridges requiring lane and shoulder 
closures to be identified and specified in bridge 
inspection assignments carried out by consultants. 
The Ministry's regional structural engineers are 
now required to make an accessibility assessment 
for each bridge before advertising the assignment. 
The required number of lane and shoulder closures 
is communicated to the consultants on the basis of 
these assessments. 

In March 2010, the Ministry developed its 
"Bridge Inspection Accessibility Guidelines" to be 
used by regional structural engineers to develop 
accessibility plans for their bridges. The plans 
include information on access requirements, such 
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as lane and shoulder closures; special access equip- 
ment required, such as bucket trucks and boats; 
suspect areas that require an enhanced inspection; 
and the frequency of the enhanced inspections. The 
Ministry indicated that in April and June 2010 it 
delivered workshops to all ministry inspectors on 
completing the accessibility plans. 

At the time of our follow-up, the Ministry 
informed us that the 2009 inspection cycle had 
required 50 lane and shoulder closures, and the 
2010 inspection cycle had required 100. 

INSPECTION OVERSIGHT 

Recommendation 4 

To ensure that inspections are conducted in accord- 
ance with legislation, the Ministry of Transporta- 
tion should establish a risk-based approach for the 
ongoing monitoring of inspections. This approach 
should include: 

• assessing the reasonableness of the number of 
bridges that external contractors and ministry 
staff report as having been inspected in any one 
day to ensure that thorough inspections are 
being done; 

• following up on any unusual changes in a 
bridge's condition since the previous inspection; 
and 

• identifying high-risk bridges that should be 
subject to more in-depth condition surveys. 

The Ministry of Transportation should abo 
consider standardizing its agreements with engineer- 
ing firms. At a minimum, these agreements should 
contain provisions regarding the experience and 
qualifications of staff assigned by the firm to conduct 
the inspections. 

Status 

At the time of our follow-up, the Ministry indicated 
that the September 2009 policy memo requires that 
regional structural engineers and project managers 
record an estimate of the minimum inspection time 
for each bridge, and that estimate is then compared 
to the actual time taken to inspect the bridge. This 



is intended to ensure that the inspector has taken 
the appropriate amount of time to complete the 
inspection. The 2010 bridge inspection workshops 
conducted by the Ministry also addressed con- 
sistency in time spent on the inspection process. 
The Ministry also noted that the BMS has added 
new data fields that record the time required for 
inspections and the start- and end-dates of the 
inspections. 

Ministry policies and procedures now require 
that the regional structural engineer must review 
any bridges with significant changes in condition 
(either an increase of more than three points or a 
decrease of more than five points in BCI values) and 
that any such change be justified and rationalized 
by the bridge inspector. The Ministry also indicated 
that a new data field has been added to the BMS 
requiring an explanation for unexpected changes in 
the BCI between inspections. 

The Ministry has also developed a standard 
Request for Proposal (RFP) document for inspec- 
tions that are outsourced to engineering firms. 
The RFP now requires all lead inspectors to have a 
minimum of five years' inspection experience and 
to have completed the Ministry's bridge inspection 
course. 

BRIDGE MAINTENANCE 

Recommendation 5 

The Ministry of Transportation should: 

• develop a formal asset-management plan as 
a basis on which to prioritize the preventative 
maintenance of bridges; and 

• promptly carry out preventative maintenance, 
including the maintenance recommended in 
bridge inspections. 

Status 

At the time of our follow-up, the Ministry indicated 
that it had instituted an interim process for tracking 
maintenance work. This process: 

• defines the urgent maintenance needs that 
may affect safety; 
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• requires that items with the highest priority 
be completed first and prioritizes the remain- 
der; and 

• includes an annual chart to list non-urgent 
maintenance items for all bridges and record 
all maintenance work completed, with the 
information to be returned to the sections 
responsible for structures within each region. 

The Ministry also informed us that its multi-year 
regional investment plans (discussed previously) 
include information on optimal preventative main- 
tenance work required and the impact this work 
has on extending the life of structures. Planned 
improvements to the BMS will allow the Ministry to 
create reports on required maintenance work that 
can be distributed to regional offices. 

ONTARIO BRIDGE MANAGEMENT 
SYSTEM 

Recommendation 6 

To make the Ontario Bridge Management System 
more useful, the Ministry of Transportation should: 

• ensure that the information on bridge rehabili- 
tation contained in the System is up to date; and 

• assess whether the System meets users' needs 
and whether there are cost-effective ways of 
improving its performance and capabilities, 
especially with respect to reporting informa- 
tion needed for rehabilitation and inspection 
purposes. 

Status 

At the time of our follow-up, the Ministry indicated 
that it had updated the rehabilitation history for all 
bridges and transferred the data to the "Work His- 
tory" section of the BMS. 

The Ministry also informed us that it has made 
a number of improvements to its BMS, focusing 
on improving overall system performance as well 
as data access and reporting capabilities. Ministry 
documentation indicated that the overall speed 
of operation of the BMS over the Ministry- wide 
network has improved. The BMS now allows easier 



management of bridge-related documents, such as 
engineering drawings and inspection photographs 
and reports. Engineering drawings that were 
previously contained in a separate document man- 
agement system have been loaded into the BMS 
database. 

The Ministry indicated that it has started the 
development of a plan to replace the current BMS. 

PROCUREMENT AND CONTRACT 
MANAGEMENT 

Recommendation 7 

To ensure value for money on major capital projects 
and fairness in its procurement process, the Ministry 
of Transportation should: 

• review the application of its two different sets of 
evaluation criteria for requests for proposals to 
ensure that they are consistently applied across 
the regions; 

• reassess the evaluation criteria in which the bid 
price is a relatively minor factor in selecting the 
winning bidder; and 

• given the frequent significant variances between 
the Ministry's estimated cost of a project and 
the bidder's cost, examine its internal estima- 
tion process as well as the possible impact of the 
increased trend of relatively few bidders. 

Status 

At the time of our follow-up, the Ministry informed 
us that it plans to explore an approach where 
simple and straightforward engineering assign- 
ments may be awarded solely on price, while for 
larger and more complex projects it would also 
consider factors such as the consulting engineering 
firm's past performance and proposed approach to 
the work. 

The Ministry also indicated that it has been 
monitoring the variance between the estimated and 
the bidder's cost of design projects and has seen 
some improvement in this regard. The Ministry 
informed us that it will continue to monitor the esti- 
mated and actual costs of design projects, and if the 
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average actual costs exceed the estimated costs by 
5%, it will analyze the reasons for the variance and 
implement measures in 2012 to further improve its 
cost estimates. These measures would include using 
the detailed breakdown of the bids of past design 
projects to estimate future design project costs. 

To address the increased trend of relatively few 
bidders, the Ministry indicated that it continues to 
meet with senior members of the consulting indus- 
try to identify opportunities to increase interest in 
its engineering assignments. 

MUNICIPAL BRIDGES 

Recommendation 8 

To help ensure the safety and proper upkeep of 
municipal bridges, and as part of its current 
provincial-municipal review, the Ministry of Trans- 
portation should work with municipalities and other 
stakeholders to: 

• review practices in other large provinces and 
U.S. states with respect to oversight of municipal 
responsibilities for bridge maintenance, with the 
aim of determining whether changes to the cur- 
rent accountability relationship are required; 

• ensure that the condition of municipal bridges 
is consistently assessed, updated every two years 
as required, and publicly reported; 

• review the Ministry's funding arrangement with 
municipalities to ensure that the funds provided 
are effective in sustaining the proper mainten- 
ance and rehabilitation of bridges; and 

• promote good asset-management practices. 

Status 

In fall 2009, the Ministry, in conjunction with the 
Association of Municipalities of Ontario and the 
City of Toronto, launched a joint provincial- 
municipal review to develop options for respon- 
sibilities and funding arrangements for municipal 
roads and bridges. A steering committee and three 
working groups have been established to: 



• promote the development of asset manage- 
ment plans for municipal roads and bridges; 

• develop objective criteria and a methodology 
for evaluating the municipal road network 
and determining which municipal roads are 
of provincial, municipal, or joint provincial- 
municipal interest across Ontario; and 

• develop a funding framework for municipal 
roads and bridges that considers municipal- 
ities' investment needs and their ability to 
fund the required infrastructure. 

The Ministry indicated that the final recommen- 
dations from the working groups were anticipated 
in fall 2011. 

At the time of our follow-up, the Ministry also 
indicated that a review of municipal bridge over- 
sight practices in most Canadian provinces and 
some U.S. states was under way, and the informa- 
tion gathered would be considered in assessing 
options for municipal bridge oversight in Ontario. 
Final recommendations from this jurisdictional 
review were also anticipated in fall 2011. 

At the time of our follow-up, the Ministry indi- 
cated that it continues to provide funding to assist 
smaller municipalities to collect data on the condi- 
tion of bridges and input the data in Municipal 
Data Works (MDW), a web-based system designed 
to manage municipal tangible capital assets. MDW 
was developed using the Ontario Structure Inspec- 
tion Manual's method of conducting bridge inspec- 
tions. This involves dividing the bridge into 20 to 30 
elements and determining the quantity of defects 
in each element measured by four condition states. 
Using the inspection results, MDW can also deter- 
mine the BCI for individual bridges. The system 
can also store information on the maintenance and 
rehabilitation needs of individual bridges. 

According to the Ministry, as of June 2011, 
approximately 70% of municipalities either had 
loaded or were in the process of loading their 
bridge data into MDW. 
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Ministry of Consumer Services 

Sectionl 

4.03 Consumer Protection 



Follow-up on VFM Section 3.03, 2009 Annual Report 




^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

The Ministry of Consumer Services (Ministry) over- 
sees business and industry practices in Ontario's 
consumer marketplace for the protection of con- 
sumers and public safety. It does this by educating 
the public and businesses, responding to complaints 
from the public, monitoring and inspecting busi- 
nesses, and enforcing compliance with a number of 
consumer protection regulations and laws, such as 
the Consumer Protection Act, 2002. 

In the 2010/11 fiscal year, the Ministry carried 
out these responsibilities with a staff of about 113 
(110 in 2008/09) and operating expenditures of 
approximately $12.8 million ($12.6 million in 
2008/09). 

The responsibility for certain marketplace 
sectors has been delegated to eight designated 
administrative authorities (delegated authorities), 
which are not-for-profit corporations (see Figure 1). 
The corporations are funded by the industries they 
oversee and use their industry and technical exper- 
tise to protect the public. The Ministry monitors 
the performance and activities of these delegated 
authorities. 

At the time of our 2009 audit, we noted that the 
Ministry had made progress in addressing many of 
the recommendations in our 2003 audit, including 
recent improvements to its oversight of delegated 
authorities. Several changes to legislation had 



also strengthened consumer protection, and the 
Ministry had carried out initiatives to promote 
compliance with consumer protection legislation by 
certain industries. Nonetheless, we noted that cor- 
rective action was required in the following areas: 

• The Ministry needed to better promote its 
mandate and services to consumers. The 
almost 40,000 inquiries and written com- 
plaints it received in the 2008/09 fiscal year 
represented a 12% drop in volume from peak 
levels in 2004/05, but the Ministry had done 
no work to assess the reasons for this decline. 
Quebec's consumer protection agency, by 
comparison, received more than 250,000 
consumer inquiries and complaints annually. 
In addition, our own independent external 
survey indicated that the Ministry would not 
be among Ontarians' top choices for resolving 
a complaint. 

• The Ministry needed to deal more effectively 
with problem industries and repeat offend- 
ers, such as collection agencies, which had 
consistently been on the Ministry's Top 10 
Complaints list from 2000 through 2008. 

As well, limited inspection staff resources 
resulted in no proactive visits during 2008/09 
to the types of businesses in the Top 10 Com- 
plaints list, and the Ministry initiated only 148 
inspections and educational field visits as a 
direct result of the 6,000 written complaints it 
received. In addition, the Consumer Protection 
Act, 2002, which covers the vast majority of 
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Figure 1: The Ministry's Delegated Authorities 

Source of data: the most recent published annual report of each delegated authority 









Annual 




Delegated Authority 






Expenditures 


#of 


(Year Established) 


Primary Legislation Administered and Key Responsibilities 




($ million) 


Staff 


Board of Funeral Services 

(BoFS) 

(1914) 


Funeral Directors and Establishments Act 

regulates funeral services; licenses 3,300 businesses and 
inuiviuuais 


1.8 


10 


Electrical Safety Authority 

(ESA) 

(1999) 


Electricity Act, 1998 

regulates the use of electricity and electrical equipment; enforces 

lllc UNldNU LIcOLIIGdl odlcly l^UUc, IICclloco lllOic UldN 1/,UUU 

electrical contractors and electricians 


85.2 


482 


Ontario Motor Vehicle 
Industry Council (OMVIC) 
(1997) 


Motor Vehicle Dealers Act, 2002 

regulates about 7,800 motor vehicle dealers and 24,000 
salespersons 


10.0 


78 


Real Estate Council of 
Ontario (RECO) 
(1997) 


Real Estate and Business Brokers Act, 2002 

regulates 60,000 real estate brokerages, brokers, and salespersons 


10.7 


87 


Tarion Warranty Corporation 

(Tarion) 

(1976) 


Ontario New Home Warranties Plan Act 

administers a mandatory new home warranty program; registers 
5,400 builders; enrolled more than 1.5 million homes 


20.7 


250 


Technical Standards and 
Safety Authority (TSSA) 
(1997) 


Technical Standards and Safety Act, 2000 

licenses, registers, and certifies about 115,000 devices, facilities, 
contractors, workers for boilers and pressure vessels, operating 
engineers, amusement and elevating devices, fuels, and 
upholstered and stuffed articles industries 


45.5 


340 


Travel Industry Council of 
Ontario (TICO) 
(1997) 


Travel Industry Act, 2002 

regulates about 2,500 travel retailers and wholesalers 


4.0 


23 


Vintners' Quality Alliance of 
Ontario (VQA Ontario) 
(2000) 


Vintners Quality Alliance Act, 1999 

regulates VQA standards for more than 120 registered wineries 


1.2 


3 



businesses, does not give the Ministry certain 
powers that inspectors in other Ontario 
government programs and other provinces 
typically receive, and the lack of these powers 
hindered the Ministry from identifying and 
remedying consumer protection violations. 
• The Ministry had made some progress since 
our previous audit in enforcing compliance by 
cemetery owners with reporting requirements 
under the Cemeteries Act. However, we identi- 
fied a number of financial discrepancies that 
the Ministry had not followed up on. 



• The Ministry had launched a comprehensive 
review of delegated authorities on an urgent 
basis only after a tragic propane explosion 
in Toronto on August 10, 2008. Four years 
earlier, the Standing Committee on Public 
Accounts had recommended that the Ministry 
conduct such a review. 

• The boards of directors of delegated author- 
ities were dominated by representatives of the 
industries they regulate. The Ministry had not 
encouraged more representation from gov- 
ernment, consumers, and the public on such 
boards. 
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• Despite the Ministry's responsibility to oversee 
the delegated authorities, the Ministry had no 
right to access delegated authorities' informa- 
tion on matters such as quality assurance pro- 
grams, strategic plans, executive salary and 
compensation packages, and board minutes. 

• We noted that only one performance measure 
was reported publicly to cover all consumer 
protection programs delivered directly by the 
Ministry, and we questioned whether it was a 
reliable and meaningful measure. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 



Status of Actions Taken on 
Recommendations 



According to information we received from the 
Ministry, some progress has been made in address- 
ing all of our recommendations, with significant 
progress being made on several. The Ministry 
researched how other provinces promote their con- 
sumer protection mandates and services, and what 
enforcement measures they use, in order to consider 
whether these could be incorporated into its own 
practices. The Ministry also has made improve- 
ments to its information systems and internal 
processes to better identify businesses with a history 
of violations. We recognize that the Ministry will 
need additional time to fully address several of our 
recommendations and that the Consumer Protection 
Act, 2002 limits its inspection and enforcement pow- 
ers over businesses that are regulated under this Act. 
The status of action taken on each recommendation 
at the time of our follow-up was as follows. 



CONSUMER PROTECTION PROGRAMS 
DELIVERED DIRECTLY BY THE MINISTRY 

Public Awareness of the Ministry's 
Mandate and Consumer Protection 
Legislation 

Recommendation 1 

To ensure that there is adequate public awareness 
of the Ministry's consumer protection mandate and 
complaint services, the Ministry should: 

• consult with other jurisdictions that have 
significantly more activity and recognition by 
the public to see if there are any best consumer- 
protection practices that can be applied in 
Ontario; 

• assess its outreach and education programs with 
a view to identifying changes needed to make 
them more effective; and 

• establish mechanisms for regularly assessing the 
general public's awareness of consumer rights 
and the Ministry's programs. 

Status 

The Ministry informed us that it had conducted 
comprehensive research of public outreach and 
awareness practices in other Canadian jurisdic- 
tions. The Ministry had identified best practices 
and was considering them for possible adoption 
in Ontario. For example, the Ministry noted that 
Internet applications that facilitate information 
sharing, such as blogs and social networking, are 
useful tools for informing and communicating with 
young consumers. 

The Ministry informed us that a media relations 
campaign began in January 2010 to raise aware- 
ness of the Ministry and to inform and educate 
consumers about their rights and responsibilities. 
The Ministry said the campaign involved about 650 
public outreach initiatives, including media inter- 
views and other events, and covered topics such as 
moving companies, home renovations, and motor 
vehicle repairs. 

The Ministry acknowledged that three ministry 
realignments and name changes in the recent past 
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had likely had the effect of lowering the Ministry's 
public profile. However, in June 2009, the Min- 
istry of Consumer Services became a stand-alone 
ministry, and it created a new website in October 
of that year. Consumer use of the new website has 
increased steadily — more than 540,000 visits dur- 
ing 2010 and 235,000 during the first three months 
of 2011, compared to the 7,800 and 28,000 visits 
during 2007 and 2008, respectively, that we noted 
in our 2009 report. Twenty-one new informational 
videos were also created and posted to the Ministry 
website. 

The Ministry also reports some improvement 
to its service volumes: it received 40,000 phone 
inquiries and 7,300 written complaints in 2010/11, 
an increase of about 19% from 2008/09. 

The Ministry commissioned a survey in fall 2010 
and market research analysis in winter 2011 to 
gauge the public's awareness of consumer protec- 
tion and the Ministry. The survey results largely 
confirmed the results of our 2008 survey — that the 
public did not regard the Ministry as a top source 
of help on consumer issues. The Ministry told us 
it intended to compare results from future surveys 
against this one so it could measure its progress in 
increasing awareness of consumer protection legis- 
lation and the Ministry's services. 

Since fall 2010, the Ministry initiated, com- 
pleted, and began implementing a long-term Con- 
sumer Awareness Strategy. The strategy includes 
goals for educating the public about their consumer 
rights and responsibilities, responding to consum- 
ers' concerns and issues promptly, and increasing 
public awareness of the Ministry as a source of 
consumer information and protection. The strategy 
also includes activities for achieving these goals and 
measuring its success. 

Registration and Licensing 

Recommendation 2 

To ensure that its registration processes meet 
legislative requirements that only businesses that 
demonstrate financial responsibility and honesty and 



integrity are registered and licensed, the Ministry 
should: 

• review the procedures, documentation require- 
ments, and quality control processes that its 
staff must follow to conduct a proper and com- 
plete review of an application; and 

• establish a policy and guidelines for staff to 
use that would require due consideration of 
the number and types of complaints about an 
applicant. 

Status 

The Ministry informed us that it had reviewed and 
updated its registration and licensing procedures, 
including documentation requirements, for collec- 
tion agencies, cemeteries, payday loan units, and 
bailiffs. Checklists for these sectors were developed 
and implemented to help ensure that staff reviewed 
applications in a consistent manner. 

The Ministry advised us that if the number 
and type of complaints about a particular licensed 
business do not entitle that business to continue 
to be registered and operate, steps to suspend or 
revoke the registration would be taken at the time 
the determination is made, rather than delayed 
until, for instance, the registration renewal date. 
However, the Ministry informed us that no business 
licences had been revoked or suspended in the last 
two years as a result of a business's history of viola- 
tions, or for any other reasons. 

The Ministry implemented changes to its Con- 
sumer Affairs Tracking System (CATS), which is 
used by staff to record inquiries or complaints by 
consumers and any response or action taken by the 
Ministry. The Ministry told us the changes would 
improve monitoring by tracking issues and flagging 
violators whose conduct might call for direct min- 
istry intervention. 

The Ministry has also created the position 
of Complaint and Compliance Analyst, and that 
person is to take proactive measures to address 
non-compliance by payday loans companies and 
collection agencies. 
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Dealing with Consumer Complaints 

Recent Initiatives and Information Systems 
Recommendation 3 

To enhance the ability of staff to use the information 
recorded in the Consumer Affairs Tracking System to 
analyze consumer issues by the type of industry and 
the type of inquiry or complaint, the Ministry should: 

• capture information on its inspections and 
educational field visits by industry and violation 
type and on the type of information providedfor 
the public inquiries; and 

• ensure that the nature of all inquiries and calls 
is input into the system. 

Status 

The Ministry informed us that in winter 2010 it 
enhanced the Consumer Affairs Tracking System 
(CATS) in a number of ways: 

• Fields have been added to the inspections 
modules so that staff can record the nature of 
the inspection or educational field visit and 
the type of industry and service inspected. 

• Data verification checks help ensure that 
inspectors enter a description of the deficien- 
cies, as well as other required information. 

• New reporting capabilities monitor the 
outcome of all written and verbal complaints 
received about collection agencies and payday 
loan companies for a particular time period. 

We were also advised that performance 
appraisal forms for those staff whose jobs include 
receiving complaints from the public were revised 
to include an evaluation of the employee's docu- 
mentation of the complaints and actions taken. 

Problematic Industries and Repeat 
Offenders 

Recommendation 4 

To ensure that it can effectively deal with industries 
and businesses that incur high numbers of and/or 
repeated consumer complaints, the Ministry should: 

• conduct research to identify best practices in 
other provinces that can be applied in Ontario 



to improve compliance by certain industries and 
businesses; and 
• identify industries and businesses that per- 
sistently incur a high number of consumer 
complaints, assess the effectiveness of its past 
enforcement activities used against these prob- 
lematic industries and businesses, and establish 
effective education and enforcement strategies 
for dealing with them. 

Status 

The Ministry informed us that from July 2010 to 
May 2011 it conducted a survey of compliance tech- 
niques and best practices in use in other provinces. 
It said it plans to use the results, which it summar- 
ized in a June 21, 2011 report, to identify strategies 
that could improve the compliance and enforce- 
ment function in Ontario. 

The Ministry advised us that it had updated its 
Investigation Priority Assessment System (IPAS), 
which annually assigns a relative priority to files 
that are flagged for investigation, as well as its 
annual inspection allocation review process, which 
designates the industries in which inspections and 
field visits will be made. Due to limited staff resour- 
ces, the Ministry investigates cases with the poten- 
tial for higher risk of financial loss to consumers 
and/or a greater number of complaints and victims. 
Higher priority is also assigned to those industries 
identified by government as requiring more atten- 
tion because of the risks they pose to society and 
more vulnerable people. The Ministry informed 
us that compliance and enforcement resources 
have focused on industries with high complaint 
volumes, such as the water heater rental industry, 
home furnishing movers, fitness clubs, and home 
renovations. 

The Ministry retained a consulting firm in spring 
2011 to review the operations and organizational 
structure of its Consumer Protection Branch 
(Branch) . The review determined that for the Min- 
istry to ensure that it applied its resources in the 
most effective way, its risk-based approach needed 
to be better defined and embedded in the processes 
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that govern the way work is prioritized and resour- 
ces allocated. It also found that the Ministry needed 
to increase its educational and outreach activities 
for businesses and consumers. 

As well, the Ministry informed us that a number 
of steps have been taken to improve the Branch's 
ability to respond to identified risks. Additional 
resources have been allocated to strengthen its 
investigation and enforcement program; IPAS was 
updated and amended to better identify risks, pri- 
orities, and appropriate enforcement mechanisms; 
complaint volume statistics are regularly monitored 
to identify sectors where targeted compliance 
action or communications may be warranted; and 
modifications to CATS have made it easier for the 
Ministry to access and analyze relevant data. 

Inspections and Educational Field Visits 
Recommendation 5 

To expand its coverage and capabilities for its inspec- 
tion activities for the protection of consumers, the 
Ministry should: 

• conduct a formal assessment of the number 
of inspection staff resources it should have to 
adequately fulfill its mandate and ensure com- 
prehensive coverage; and 

• explore the need to obtain increased legislative 
authority and powers for its inspectors, consist- 
ent with those in other consumer protection 
organizations in Ontario and other provinces, 
that would allow them to more efficiently and 
effectively deal with consumer complaints 
and identify potential consumer protection 
violations. 

Status 

We noted that the Ministry had been able to increase 
the number of inspections and educational field 
visits since our 2009 audit from 352 in 2008/09 to 
500 in 2010/11 with the addition of inspection staff, 
as well as a ministry priority to increase inspections 
of payday loan businesses and of cemetery owners 
who failed to submit annual filings. 



The Ministry engaged the Ministry of Commun- 
ity and Social Services to prepare an inspection- 
staffing analysis, which was completed March 31, 
2011. The analysis concluded that the Ministry had 
an adequate level of inspection staff for the current 
volume of requests based on having six inspection 
staff, including two new staff hired since our 2009 
report. However, the analysis only considered the 
staff resources needed to address internal requests 
for inspections during 2010; it did not assess 
whether sufficient inspections were being done 
to fulfill the Ministry's regulatory mandate and 
encourage voluntary compliance. 

Following is a recent example that illustrates 
the need to have adequate inspection resources. 
The Ministry made arrangements for a pilot project 
in which field service officers from a Ministry of 
Revenue branch office would conduct inspections 
or educational field visits for Consumer Services 
concurrent with their own Revenue inspections. 
From February 2009 to January 2010, the field 
service officers conducted 23 inspections under the 
Film Classification Act, 2005 of stores that sell DVD 
videos and 32 field visits under the Consumer Pro- 
tection Act, 2002 of motor vehicle repair facilities. In 
all of these cases, the field service officers identified 
violations, primarily regarding unlicensed premises 
selling videos and repair facilities not providing 
required disclosure to consumers on their signs. In 
follow-up visits, the Ministry of Consumer Services 
was able to obtain compliance from all the video 
vendors, but 29 of 32 motor vehicle repair facilities 
still had not complied after the follow-up visit. The 
pilot project with the Ministry of Revenue ended, 
and there are no current plans to resume the 
project. 

The Ministry noted that the Ministry of 
Community and Social Services had developed 
an inspection model that could be used by the 
Branch to more effectively allocate its compliance 
and inspection resources to target the identified 
priorities, including the top 10 complaints the 
Branch receives. In addition, the Ministry said that 
implementation of the recommendations from the 
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spring 2011 review of the Branch's operations and 
organizational structure would also strengthen its 
risk-based approach for allocating resources and 
more effectively focus educational and outreach 
activities. 

The Ministry also told us it was engaged in a 
partnership with the Ministry of the Environment 
to use its inspectors to identify contraventions of 
the Consumer Protection Act, 2002 relating to waste 
diversion fees (known as eco fees). 

In our 2009 report, we noted that consumer 
protection legislation in other provinces gives their 
inspectors powers to enter business premises at any 
reasonable time and to inspect, audit, or examine 
any record, goods, or services in the premises, 
and take copies if needed. The Ministry does have 
the authority to conduct inspections of registered 
businesses under powers outlined in specific pieces 
of legislation, such as legislation pertaining to 
payday loans and collection agencies. However, 
the Consumer Protection Act, 2002, which covers 
the vast majority of businesses, does not provide 
such authority; for these businesses, the Ministry 
conducts "educational field visits" without the 
authority to request or inspect books or records. 
The Ministry said it considered our recommenda- 
tion and concluded that inspection power under the 
Consumer Protection Act, 2002 was not necessary 
and would not be appropriate because it would be 
a disproportionate intervention when applied to 
the broad retail community, and would raise legal 
risks in its implementation. Instead, the Ministry 
determined that a "document production power" 
to require businesses to provide it with copies of 
documents they use when dealing with consum- 
ers — things like contracts, application forms, 
monthly bill statements and advertising — would 
be more appropriate, although the documents 
would only need to be sample documents and not 
the documents actually completed with consumers. 
The Ministry indicated that a timetable for making 
changes to the legislation, if any, would depend on 
government approval. 



Investigations and Enforcement 
Recommendation 6 

To help ensure that the Ministry's enforcement efforts 
are both timely and cost-effective in achieving compli- 
ance and in deterring future violations of consumer 
protection laws, the Ministry should: 

• consider introducing more expeditious and 
effective enforcement tools, including admin- 
istrative monetary penalties and tickets, for 
violations that either do not warrant criminal 
prosecution or are less serious; and 

• undertake periodic reviews, including research- 
ing best practices in other similar organizations, 
of its investigative program, enforcement meas- 
ures, and the Consumer Beware Database, to 
assess their effectiveness and identify areas for 
improvement. 

Status 

In our 2009 report we noted that between 2002/03 
and 2008/09, there were some large declines in the 
Ministry's investigation activities and enforcement 
outcomes. As Figure 2 indicates, the results have 
been mixed over the past two years. 

The Ministry informed us that the Payday Loans 
Act, 2008, which requires registration of payday 
lenders and loan brokers, is the only statute that 
includes provisions for imposing administrative 
monetary penalties. A regulation allowing penalties 
of $100 to $3,000 for violations came into force in 
July 2009. At the time of our follow-up, the Min- 
istry was preparing a policy framework to be used 
to ensure consistent application of these penalties 
for violations by payday loan companies. The 
Ministry advised us that during the initial round 
of inspections after the regulation came into force, 
and in keeping with the Ministry's progressive 
approach to compliance and enforcement, inspect- 
ors issued almost 120 cautions to businesses about 
contraventions. The Ministry plans to conduct 
follow-up visits to establishments that received cau- 
tions. If contraventions persist, and if the inspect- 
ors, exercising their statutory discretion, believe it 
is appropriate, administrative monetary penalties 
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Figure 2: Investigation Activities and Enforcement Outcomes by the Ministry's Consumer Protection Branch, 
2008/09-2010/11 

Source of data: Ministry of Consumer Services 







Changes between 
2008/09 and 




2008/09 


2009/10 


2010/11 


2010/11 (%) 


# of investigations closed 


114 


131 


91 


(20) 


# of individuals and businesses charged 


158 


182 


124 


(21) 


# of convictions 


161 


257 


320 


100 


length of jail time and probation ordered by courts (months) 


474 


606 


787 


66 


settlements negotiated by investigators prior to prosecution* ($) 


100,283 


32,812 


72,127 


(28) 


restitution ordered by courts ($) 


327,656 


673,464 


430,103 


31 


amount of court fines levied ($) 


384,850 


324,500 


317,800 


(17) 



* This excludes refunds obtained each year through the Ministry's complaints-handling process and mediation services prior to any investigations ($437,645 in 
2008/09; $334,052 in 2009/10; $330,099 in 2010/11). 



will be issued. (An inspector issued a monetary 
penalty in April 2011 as a result of deficiencies 
noted from a follow-up visit.) We were also advised 
that as more payday loan company inspections 
are conducted, the reasons for issuing administra- 
tive monetary penalties and their effectiveness in 
achieving compliance will be monitored. 

The Ministry told us it considered the possibility 
of issuing tickets under the Provincial Offences Act 
but rejected this approach — primarily because, 
under that legislation, it could not issue a ticket 
that exceeded $1,000, and it felt that violations of 
consumer law, which are largely motivated by the 
prospect of financial gain, require heavier penalties 
to promote compliance. 

As previously mentioned, the Ministry con- 
ducted a recent survey of compliance techniques 
and best practices in other jurisdictions and intends 
to review the results to see if they are applicable 
in Ontario. In November 2010, the Ministry also 
completed research into what information other 
jurisdictions' consumer protection offices disclosed 
through on-line postings. This resulted in several 
recommendations on ways to improve disclosure 
through the Ministry's Consumer Beware Database 
on its website. The Ministry advised us that an 
amendment to the regulation under the Consumer 
Protection Act, 2002 would be required for addi- 



tional information to be published and that a time 
frame for such a change is dependent on govern- 
ment approval. 

Cemeteries' Trust Accounts 

Recommendation 7 

To ensure that cemetery owners comply with legis- 
lative reporting requirements and that funds are 
accounted for and sufficient for the proper long-term 
care and maintenance of cemeteries, the Ministry 
should ensure that: 

• all annual returns are filed by all cemetery 
owners; and 

• timely and effective action is taken to enforce 
reporting requirements, to properly assess 
reports received, and to follow up on and resolve 
financial discrepancies identified on returns. 

In view of the significant demand that cemetery 
legislation places on the Ministry's limited staff 
resources, the Ministry should also explore the option 
of having cemetery legislation administered by a dele- 
gated authority. 

Status 

The Ministry has made progress in improving and 
tracking how well cemetery owners comply with 
their reporting requirements. We were advised 
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that the overall compliance rate for the filing of 
2009 annual returns was 92%. As of August 2011, 
the Ministry had received 82% of annual returns 
for 2010. In addition, 97% of trust agreements for 
large cemeteries (those deemed to pose a higher 
financial risk) had been submitted and reviewed. 
Those agreements that had not been submitted 
were being tracked. The Ministry mailed to cem- 
etery owners an updated guide and instruction 
sheet with annual return forms in November 2010. 
The Ministry expected that by the end of the 2011 
calendar year the percentage of those in compli- 
ance on filing 2010 annual returns would be similar 
to that of 2009. 

The Ministry advised us that it implemented 
changes to the CATS database to flag trust fund 
variances and capture information related to trust 
fund agreements and cemetery trust fund deficien- 
cies. The Ministry said enhancements to CATS 
now support its processes to ensure that annual 
statements are reviewed and reconciled and mon- 
ies are deposited by cemetery owners within the 
prescribed time limits into care and maintenance 
trust funds. 

The Ministry had not yet explored the option 
of having cemetery legislation administered by a 
delegated authority. 

MINISTRY OVERSIGHT OF DELEGATED 
AUTHORITIES 

Recommendation 8 

To better protect consumers and the public, the Min- 
istry should strengthen its oversight role and account- 
ability arrangements with designated administrative 
authorities (delegated authorities) by: 

• establishing formal comprehensive accountabil- 
ity agreements with each delegated authority 
that cover financial and operational require- 
ments and that would protect the public's 
interests; 

• encouraging a more appropriate and fair bal- 
ance of representation on boards of directors 



between governments, consumers, the public, 
and industry; 

• ensuring that it has the necessary authority 
over delegated authorities to access any relevant 
information needed, such as information on 
quality assurance programs and use of financial 
resources, that would allow for a comprehensive 
and thorough assessment of their financial and 
operational performance, and where the Min- 
istry's authority to do so is in question or lim- 
ited, seeking the legislative changes necessary to 
grant it unfettered authority in this regard; and 

• establishing requirements that delegated 
authorities provide consistent performance 
information and compare their performance to 
similar organizations in other jurisdictions. 

Status 

We were informed by the Ministry that it entered 
into a new memorandum of understanding with the 
Technical Standards and Safety Authority (TSSA) 
in April 2010 that includes additional corporate 
reporting requirements and better information- 
sharing protocols, and that reiterates the new audit 
provisions in the amended Technical Standards 
and Safety Act. The Ministry also established a 
new accountability agreement with the Tarion 
Warranty Corporation (Tarion) in November 2010 
that includes additional corporate reporting and 
information-sharing requirements. At the time 
of our follow-up, the Ministry was in discussions 
for revised administrative agreements with the 
Real Estate Council of Ontario (RECO), the Travel 
Industry Council of Ontario (TICO), the Ontario 
Motor Vehicle Industry Council (OMVIC), and the 
Electrical Safety Authority (ESA). Revised draft 
agreements had been presented to the authorities, 
and the Ministry expected new agreements to be 
in place by fall 2011. It further advised us that 
accountability requirements for the Board of 
Funeral Services are incorporated into new legisla- 
tion expected in 2012 that will govern this sector. 

The Ministry had hired a consultant to con- 
duct research on other jurisdictions' practices 
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on establishing boards of directors for delegated 
authorities. The study was expected to be com- 
pleted in September 2011 and was to identify best 
practices for board appointments, composition and 
skills, committee structures, and term limits. The 
Ministry also increased the percentage of board 
members it appoints on two authorities — from 
29% to 46% for the TSSA, a planned increase that 
had been decided at the time of our 2009 audit 
and that followed a tragic propane explosion in 
August 2008; and from 27% to 33% for the TICO, 
following the sudden bankruptcy of a large travel 
retailer in April 2009. Ministry-appointed represen- 
tation on the other boards remains as it did at the 
time of our 2009 audit, ranging from 25% to 33%. 

The Ministry advised us that it continues to 
regularly receive and review information from all 
the authorities, including: 

• advisory committee meeting minutes; 

• business plans; 

• annual reports with audited financial state- 
ments; and 

• quarterly reports that include information 
on revenues, expenses, and staffing; inspec- 
tions and enforcements; complaints statistics 
against member businesses/individuals/ 
authority; and turnaround times for com- 
plaints and inspections. 

With the exception of Tarion, the Ministry's 
agreement with the delegated authorities specific- 
ally requires that they provide timely information 
in relation to any matter requested by the Minister. 
Tarion's accountability agreement requires it to 
provide the Ministry with an annual fact sheet that 
contains information not captured in the quarterly 
reports, and also requires that the CEO of Tarion 
and the Minister meet periodically to discuss, 
consult, and share information on the business 
operations of Tarion. 

The Ministry also reiterated that the authorities 
do not receive public funding. It is the responsibility 
of the boards of directors of each delegated author- 
ity to govern the financial and operational perform- 
ance of the authorities. Therefore, the Ministry 



does not currently request or receive board min- 
utes, audit committee reports, quality assurance 
assessments, information on consultant expenses, 
or information on executive and staff salaries and 
compensation. 

With regard to performance reporting, the Min- 
istry advised us it has compiled an inventory of the 
information that is submitted by the authorities, 
and is in the process of confirming the performance 
measures and targets that will be used to track the 
consumer protection and/or public-safety perform- 
ance of each authority. New public reporting on 
these measures and targets will commence for the 
2011/12 fiscal year. 

MINISTRY PERFORMANCE MEASURES 

Recommendation 9 

To improve accountability and its reporting on the 
extent to which it achieves its consumer protection 
mandate, the Ministry should: 

• report publicly on performance targets and 
measures for all its key activities; and 

• on a periodic basis, such as every two to three 
years, conduct independent consumer satisfac- 
tion surveys of its handling of both telephone 
and written complaints. 

Status 

At the time of our 2009 audit, the Ministry had four 
performance measures in its 2009/10 Results-based 
Plan (RbP), which was reported to the public. 
Only one of these measures covered all of the 
consumer protection programs delivered directly 
by the Ministry: customers' satisfaction with the 
Ministry's handling of consumer phone inquiries 
and complaints. The other three performance 
measures pertained to the Ministry's processes 
for providing oversight of delegated authorities. 
In its 2010/11 RbP, the Ministry's three measures 
pertaining to delegated authorities were replaced 
with one measure on authorities' satisfaction with 
the Ministry's performance in relationship manage- 
ment and policy development based on an on-line 
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survey distributed to them in January 2010. The 
RbP also contained information on results obtained 
by the Consumer Protection Branch in resolving 
issues, such as monetary recoveries to consumers 
and enforcement activities, and cemetery owners' 
compliance with the requirement that they file 
annual financial reports. 

As previously mentioned, the Ministry was final- 
izing for each delegated authority the measures and 
targets to be used to publicly track their consumer 
protection/public-safety and consumer awareness 
achievements. This information will be made cen- 
trally available through the Ministry's website. 

As required under the government's new Service 
Directive, the Ministry posted the Consumer Protec- 
tion Branch Service Standard commitment on its 
website in January 2010. The service standards cov- 



ered the time taken to resolve consumer complaints 
and the timeliness, helpfulness, and courteousness 
of service provided. The website also included the 
results for achieving its 2010 targets, which were 
mostly met or exceeded. However, the Ministry con- 
tinues to have the same staff member who handled 
the phone inquiry or complaint also conduct the 
satisfaction survey at the end of the call. 

The Ministry also commissioned a market 
research and polling firm to survey consumers 
who had previously contacted the Ministry for 
help with a consumer issue or with an inquiry. The 
March 2011 report covered key areas on how the 
consumers became aware of the Ministry, and the 
experience and satisfaction with its call centre, 
website, and information and help received. 
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Ministry of Education 

Section 

14.04 Education Quality and 
Accountability Office 

Follow-up on VFM Section 3.04, 2009 Annual Report 



Background 



The Ontario government established the Education 
Quality and Accountability Office (EQAO) as a 
Crown agency in 1996, with a mandate to develop 
and report on province-wide tests of student 
achievement. Such assessment results are intended 
to provide reliable and objective data that the 
Ministry of Education (Ministry) and the province's 
72 school boards can use to plan ways of improving 
student learning. 

Each year, the EQAO tests students in all Ontario 
publicly funded schools in Grades 3, 6, 9, and 10. 
Grade 3 and 6 students are tested in reading, writ- 
ing, and mathematics. Grade 9 students are tested 
only in mathematics. To graduate from high school, 
all students, including those in private schools, 
must pass the Ontario Secondary School Literacy 
Test (OSSLT), which is usually written in Grade 10. 
The EQAO spent $33 million in the 2010/11 fiscal 
year ($31.7 million in 2008/09), all of it funded by 
the Ministry. 

In our 2009 Annual Report, we found that the 
EQAO had adequate procedures and controls for 
ensuring that its tests accurately reflected the 
Ministry's curriculum expectations. We also noted 
that the EQAO, to ensure that the tests' level of 
difficulty was comparable between years, imposed 



strict criteria for the development and field-testing 
of questions, and thoroughly reviewed test con- 
tent. The general consensus among stakeholders, 
including principals and teachers, was that the tests 
were generally an accurate reflection of students' 
achievement in meeting the curriculum expecta- 
tions. However, we identified areas where improve- 
ments could be made: 

• The EQAO hired an external contractor to visit 
selected schools to review pre-test prepara- 
tion, ensure test booklet security, observe the 
administration of the tests, and undertake 
other quality-assurance procedures. Overall, 
the external contractor had reported a high 
degree of compliance with EQAO adminis- 
trative procedures, but an improved school 
selection process was required to reduce the 
risk of student cheating and non-compliance 
with administrative procedures. For example, 
10 of the province's 72 school boards had not 
received a visit from the external contractor 
over the previous five years with respect to the 
Ontario Secondary School Literacy Test, while 
a number of private schools with as few as five 
students taking this test were visited. 

• The EQAO must ensure that the up to 1,700 
markers it hires and trains are consistent when 
grading test papers. To do so, it seeds "validity 
papers" (test papers previously scored by an 
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expert panel) among the regular papers. The 
grades the markers give these validity papers 
are monitored to determine if retraining is 
required. The EQAO consistently met its target 
of having 95% of the validity papers graded 
within one scoring level of the expert panel's 
score. However, on a per-question basis, the 
EQAO often did not meet its target of having 
70% of the questions having the same grade 
as that given by the expert panel. 

• The EQAO informally reviews the results at 
the school and school board levels. However, 
formal analysis and follow-up may be required 
to ensure that the testing process can be used 
more effectively to evaluate the reliability of 
assessment results. For example, we noted 
that some schools' EQAO results fluctuated 
by as much as 50% from one year to the next, 
but these instances were not being systematic- 
ally flagged for follow-up to determine what 
accounted for such a dramatic change. 

• The EQAO annually reports student test 
results, as well as results from questionnaires 
given to students, teachers, and principals on 
its activities. The school staff we interviewed 
stated that the questionnaires did not suffi- 
ciently allow for feedback on ways to improve 
the testing process. They also felt that the 
EQAO should take a bigger role in explaining 
the assessment process to parents and other 
stakeholders. 

STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this audit in March 2010. In Novem- 
ber 2010, the Committee tabled a report in the 
Legislature resulting from this hearing. The report 
contained nine recommendations and requested 
the Ministry of Education, the Education Quality 
and Accountability Office (EQAO), and the Ontario 
College of Teachers to report back to the Committee 
with respect to: 



• whether the Ministry and EQAO have con- 
sidered other options concerning the matter 
where principals exempt students from EQAO 
assessments for acceptable reasons and 
then assess such students as if they had not 
achieved the provincial standard, particularly 
for those schools where a disproportion- 
ately large number of students have been 
exempted; 

• the number of exempted students over the 
previous five years, along with an explanation 
for any trends occurring over that time frame 
(EQAO); 

• a summary of the EQAO's key findings or con- 
clusions from its investigation into the matter 
of irregularities and cheating involved in last 
year's EQAO assessments (which it confirmed 
in the press), including an outline of any 
resulting policy or procedural changes it plans 
to implement; 

• what sanctions, if any, the Ontario College 
of Teachers would recommend be applied to 
teachers who have been found in violation of 
the EQAO testing rules; 

• whether the EQAO has adopted a formal pro- 
cess to investigate unusual fluctuations in test 
results and document the steps that it takes 
in dealing with these, and whether it would 
consider adding a section explaining such 
fluctuations in the standard School Reports 
that it issues to every elementary and second- 
ary school in the province; 

• the conclusions that the Ministry and the 
EQAO have drawn from the published results 
of the EQAO survey of Grade 9 teachers and 
students that asked whether using the EQAO 
assessment scores as part of the final term 
mark influences student achievement on the 
test, and whether the Ministry has considered 
a prescribed minimum percentage (as well as 
the 30% maximum) of the Grade 9 Assessment 
of Mathematics that would count as part of the 
student's final term mark; 
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• the views of the Ministry and the EQAO on 
how to best make clear to the public the differ- 
ences between the Ontario English-language 
and French-language curriculums and the 
differences in the related EQAO testing, and 
that the testing results are therefore not com- 
parable; and 

• a summary of results from the EQAO on the 
feedback it received on its pilot communica- 
tions strategy. 

The Ministry, EQAO, and the College formally 
responded to the Committee on March 30, 2011. A 
number of the issues raised by the Committee were 
similar to our observations. Where the Committee's 
recommendations are similar to ours, this follow-up 
includes the recent actions reported by the Ministry 
to address the concerns raised by both the Commit- 
tee and our 2009 audit. 







Matus ot Actions la 
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According to information received from the EQAO, 
substantial progress has been made on imple- 
menting almost all of the recommendations from 
our 2009 Annual Report. Several recommenda- 
tions — such as a more formal complaints process, 
an improved approach to selecting schools for qual- 
ity assurance visits, and a new supervisory back- 
reading process to improve marker accuracy — were 
fully implemented. At the time of our follow-up, the 
EQAO was still assessing some initiatives, such as 
on-line marker training and more open-ended ques- 
tions on its questionnaires to principals, teachers, 
and students. The status of the action taken on each 
of our recommendations is described in the follow- 
ing sections. 



TEST DEVELOPMENT AND 
ADMINISTRATION 

Recommendation 1 

To improve the Education Quality and Accountability 
Office's (EQAO's) test development and administra- 
tion process and to ensure that student assessments 
continue to be reliable and objective and that all stu- 
dents are given the opportunity to demonstrate their 
competence, the EQAO should: 

• highlight to principals and teachers any signifi- 
cant changes in the compliance requirements 
outlined in the guides to administer EQAO 
testing; 

• improve the process for selecting the schools 
visited by quality assurance monitors to ensure 
that all school boards and large private schools 
are periodically monitored; 

• assess the equity of including exempt students in 
the overall assessment results as having not met 
the provincial standard; and 

• identify schools and school boards where the 
number of exempt students appears to be rela- 
tively high and follow up to ensure that exemp- 
tions are justified. 

Status 

In its 2011 Administration Guides, the EQAO has 
added a "What's New" section that highlights the 
new procedures and information included for the 
first time this year. The EQAO has also created new 
Teacher Bulletins that summarize key administra- 
tive points relating to the process before, during, 
and after the examinations. These bulletins help to 
further emphasize key compliance requirements 
and, combined with the redesigned Administrative 
Guides, help to keep school staff informed of the 
current testing procedures. 

Following our audit in 2009, the EQAO intro- 
duced tracking procedures to help ensure that all 
school boards are visited periodically by quality 
assurance monitors during testing. The five-year 
distribution of English and French school boards 
visited has been examined to identify boards that 
may be under- or over-represented. When selecting 
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schools to visit in the current year, the EQAO will 
replace over-represented school boards with those 
in the same region that have been visited less 
often. This will allow the frequency of visits to be 
more evenly distributed over each five-year period. 
Additionally the quality assurance monitors will 
visit any schools where administrative irregularities 
or other concerns were noted in the previous year. 
At the time of our follow-up, these tracking pro- 
cedures were not in place for private schools. The 
EQAO has indicated that it will review its process to 
select large private schools for visits in future years. 

The EQAO's public website provides reports 
on each school that feature an overall assessment 
result for students who participated in EQAO 
assessments, as well as an overall result for all 
students, including those who were exempt from 
writing the tests. The EQAO is assessing options 
for reporting and displaying the results for all and 
participating students on the same page in public 
reports beginning in 2012. 

The EQAO investigates school boards that 
exceed its threshold of 6% exempt in any one type 
of assessment (reading, writing, and mathematics). 
In 2010, four English boards and one French board 
in the primary and junior levels exceeded this 
threshold for at least one assessment and therefore 
were asked for justification or to produce a plan 
to reduce their exemption rates. In addition, the 
EQAO asks all school boards to review their indi- 
vidual schools and to discuss any unusually high 
exemption rates. The EQAO has informed us that 
exemptions are not permitted for students follow- 
ing the Ontario curriculum in Grade 9 mathematics 
and the OSSLT, so that it is students participating 
in locally developed courses who fall into the 
"exempt" category. When this classification anom- 
aly is accounted for, true exemption rates fall below 
the level for concern. 



ASSESSMENT MARKING AND ANALYSIS 

Marking of EQAO Assessments 

Recommendation 2 

To improve the assessment marking process to ensure 
that results continue to be valid, consistent, and reli- 
able, the Education Quality and Accountability Office 
should: 

• consider adopting on-line training for assess- 
ment markers; 

• examine different methods to increase the num- 
ber of validity reads for each marker, especially 
early in the marking process; and 

• consider implementing supervisory backreading 
to help improve marker accuracy. 

Status 

The EQAO has informed us that it worked with a 
consultant in early 2011 to develop specifications 
for an on-line training pilot study for markers of the 
Ontario Secondary School Literacy Test (OSSLT) 
long writing question. The pilot study (conducted 
in the summer and fall of 2011) will investigate the 
feasibility of developing on-line training for mark- 
ers and implementing it in 2012. 

In 2009, the EQAO implemented a formal 
process to confirm the consistency of its marking 
by asking markers to score a set number of valid- 
ity papers per day. The EQAO tracks the average 
number of validity papers marked to determine if 
markers are meeting their daily targets. Markers of 
English-language tests are required to mark 10 valid- 
ity papers daily; there are usually no validity reads 
on the first and last day of training, when other tasks 
need to be completed. For French-language tests, 
the amount of time to score papers and perform 
other tasks including training is shorter; therefore, 
these markers may average as few as five validity 
papers daily. If markers do not demonstrate suffi- 
cient scoring accuracy, the supervisory backreading 
process is implemented. 

This process involves interventions by super- 
visors. When the scoring supervisor identifies a 
marker whose marking is not valid, as indicated 
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by the validity paper scoring reports, the marker 
receives retraining focusing on the identified prob- 
lem areas. Once retraining has been completed, the 
marker independently marks six booklets, which 
the supervisor thenbackreads. If the supervisor and 
the marker score the booklets within the acceptable 
metrics, the marker is integrated back into full scor- 
ing. However, if the scores given by the marker fall 
outside the acceptable metrics, the supervisor gives 
the marker further retraining. In this process, the 
scoring supervisor also reviews the marker's valid- 
ity reports for the following day. 

Assessment Analysis and Follow-up 

Recommendation 3 

To ensure that assessment results continue to be reli- 
able, consistent, and valid, the Education Quality 
and Accountability Office (EQAO) should enhance its 
quality assurance procedures by: 

• implementing a formal complaints process to 
help determine if there are any trends and to 
identify potential actions that could prevent 
non-compliance with assessment guidelines or 
student cheating; 

• considering more complete disclosure when test 
results at a particular school are withheld as a 
deterrent against non-compliance with assess- 
ment guidelines; 

• outlining in its administration guides potential 
penalties for violating EQAO policy; 

• tailoring its quality assurance processes to 
address unique risks associated with different 
assessments; 

• reviewing Grade 9 applied mathematics results 
to assess where incorporating EQAO results into 
the student's final mark is effective in motivat- 
ing students and, if so, suggest a more consistent 
approach; and 

• investigating any abnormally large variations in 
school assessment results from year to year and 
ensuring that they are justified. 



Status 

The EQAO continues to consult with its stakehold- 
ers to determine the process by which assessment 
irregularities can be brought to its attention by 
members of the education community and/or 
the public. Such input serves to inform the EQAO 
of ways in which it can enhance adherence to 
administration guidelines. The process being 
developed to ensure the integrity and validity of 
assessment results is expected to be posted on the 
EQAO website in the fall of 2011. While this process 
is intended to guide individuals in bringing issues 
related to assessment irregularities to the attention 
of the EQAO, the website will specify that irregu- 
larities involving students are to be brought to the 
attention of the school principal. 

In 2010, the EQAO provided the media with a 
list of schools for which results had been withheld 
because of administrative irregularities. The EQAO 
has indicated that it will continue this practice and 
has compiled a similar list for 2011. In addition, 
when a school's results are withheld because of 
administrative irregularities, the public school 
report and individual student reports indicate this 
using the symbol "W" in the place of scores and in a 
separate letter to parents. 

The EQAO's 2011 Administration Guides include 
a section describing various reporting outcomes 
that may follow if the staff identifies irregularities 
during testing. For example, no scores are assigned 
to students if they receive explanations of any 
concepts or if they are provided with instructional 
materials. The guide explains that this may have 
an impact on school and board results. The sum- 
mary of professional responsibilities distributed 
to school staff also explains that administrators 
should not encourage students in any way to revise 
their responses. Regarding this, it states that "any 
circumstances that . . . may have affected the valid- 
ity of any student performance . . . must be docu- 
mented and reported to EQAO." In March 2011, 
the EQAO developed and posted to its website the 
Video Guide to Key Administration Procedures. This 
video explains that any action taken by educators or 
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school staff to deliberately contravene test adminis- 
tration procedures constitutes professional miscon- 
duct, and states that the EQAO will refer confirmed 
cases to the Ontario College of Teachers when this 
is determined to be necessary. 

To address the increased risk of teacher or prin- 
cipal interventions in the lower grades, the EQAO 
has indicated to us that for the 2010/11 school 
year, it increased quality assurance visits to primary 
and junior assessments by 21%, while it decreased 
visits in secondary schools by 34%. EQAO monitors 
made unannounced return visits to a small number 
of schools where test administration irregularities 
were observed during their first visit. To address 
the higher risk of student collusion in high schools 
(Grade 9 and OSSLT), the EQAO selects for quality 
assurance visits all schools where an assessment of 
the pattern of responses to multiple choice ques- 
tions yields at least one "high" rating for probable 
collusion between two students. 

In 2011, the EQAO added questions to the 
teacher questionnaire that surveyed whether or not 
teachers counted the EQAO Grade 9 mathematics 
assessment as part of the students' final course 
marks. According to Ministry of Education policy, 
a final course exam must not count for more than 
30% of the course mark. Therefore, Grade 9 teach- 
ers are allowed to count the Grade 9 assessment of 
mathematics for up to a maximum of 30%. A vast 
majority of teachers indicated that they did count 
the EQAO assessment as part of the final course 
mark. Approximately 65% of students in the aca- 
demic course and 40% in the applied course said 
that their teachers would count the EQAO assess- 
ment as part of their final course mark. The number 
of students achieving the provincial standard was 
consistently higher among those who said they 
knew that the EQAO assessment would count than 
among those who did not. However, according to 
the EQAO's survey, there was no consistent pattern 
in the variation in student achievement according 
to the weight assigned to the EQAO assessment. 
The full details and results of this survey will be 
provided to teachers, principals, school board staff, 



and the Ministry of Education for consideration. 
The EQAO will advise that this information should 
be included in the course outline and assessment 
plan provided to students at the beginning of the 
school year. 

In August 2009, the EQAO began identifying 
schools with significant increases in the previous 
year in the percentage of students achieving level 3 
or above on any one assessment. A significant 
increase is defined as between 20% and 30%, 
depending on the previous year's results; a sliding 
scale is applied because it is expected that schools 
with a high percentage of students achieving the 
provincial standard are less likely to achieve a large 
gain in the following year. For the 2009/10 school 
year, 39 school boards were contacted regarding 
schools with increases that met the criteria for 
investigation. The responsibility for investigation 
lies with school superintendents, who involve 
school principals and central board staff in their 
information search. Most investigations concluded 
that improved student performance was due to 
education programs and other initiatives. The qual- 
ity assurance report prepared by EQAO noted that, 
in all cases, the schools with significant increases 
were able to provide rationales that supported the 
increases, and there was no evidence of irregular- 
ities. For its 2011 reporting, the EQAO has started 
to include guidance to help principals interpret and 
explain their school results. 

Recommendation 4 

To further improve its policies and processes and the 
procedures designed to produce accurate and reliable 
reports that can be used to improve student perform- 
ance, the Education Quality and Accountability Office 
(EQAO) should: 

• consider formalizing its pilot initiative to pro- 
vide more open-ended questions for principals, 
teachers, and students to obtain better feedback 
on any concerns with the assessment process 
and ways to improve it; 

• develop a more formal outreach strategy to give 
all schools and school boards an opportunity to 
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gain further insight into the value of EQAO data 
and how it can be used to improve student learn- 
ing; and 

• increase the understanding of parents and the 
general public of how the assessment process 
enhances student learning. 

Status 

The quality assurance monitors who visit a sample 
of schools during test administration are given 
questionnaires that contain open-ended ques- 
tions to use in their interviews of teachers and 
principals. The EQAO has indicated that resource 
constraints keep it from administering open-ended 
questionnaires directly to all principals, teachers, 
and students, but that it will consider including 
new open-ended questions each year to continue 
obtaining more feedback on its assessment process. 

The EQAO has a School Support and Outreach 
team that conducts regional workshops shortly 
after the schools receive their results. This team 
also conducts presentations on specific topics, 
profiles schools with success stories, organizes sem- 
inars, and responds to individual requests for sup- 
port by boards or schools. According to the EQAO, 
it has conducted more than 250 of these workshops 
and presentations since 2009. The EQAO indicated 
that it would enhance its outreach strategy in 2011 



by establishing a section dedicated to outreach 
on its website, giving presentations to graduating 
teacher candidates at Ontario faculties of educa- 
tion, providing opportunities for its board members 
to become involved in outreach, and using digital 
media to provide other outreach opportunities for 
educators and parents. 

The EQAO has indicated that it has developed 
a five-year parent engagement strategy to enhance 
EQAO's parent resources, increase awareness of 
those resources, build capacity for the education 
community to engage with parents about EQAO, 
and continue successful engagement activities. In 
2010, the first year of this strategy, EQAO provided 
principals with new resources and templates for 
discussing test results with parents. Opinion edi- 
torials and parent-focused communications were 
also included as part of the EQAO's media cam- 
paign. A new general information guide for parents 
was distributed in September 2010 for parents of 
elementary school students, and in spring 2011 
for Grade 9 math and OSSLT students. The EQAO 
has informed us that its board has requested the 
development of an information campaign aimed at 
the general public to be implemented during the 
2011/12 fiscal year. 
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Environment, Finance, Government Services, and Transportation 



4.05 Government User Fees 



Follow-up on VFM Section 3.05, 2009 Annual Report 




Background 



Ontario collected almost $2 billion in user fees in 
the 2010/11 fiscal year ($2.2 billion in 2008/09), 
which represents about 2% of total provincial rev- 
enues in both years. A user fee is generally charged 
to recover all or a part of the costs of providing a 
specific good or service, such as a vehicle registra- 
tion, to an individual or business that requests it. 
In contrast, a tax is used to produce revenues for 
general government purposes and for goods and 
services that the government deems to be a "public 
good," such as health care. Compared to most other 
provinces, Ontario collects less in terms of percent- 
age of total revenues obtained from user fees and 
relatively more in terms of tax revenues. 

In both the 2008/09 and 2010/11 fiscal years, 
the Ministry of Transportation collected about 
half of all user-fee revenues for driver's and carrier 
licences and vehicle registrations. Figure 1 shows 
revenues from user fees in 2010/11, broken down 
by activity and ministry or agency. In addition 
to our audit work at the Ministry of Finance, our 
audit work for the 2008/09 fiscal year focused on 
the Ministries of the Environment, Government 
Services, and Transportation, and on the Alcohol 
and Gaming Commission, which, at that time, 
accounted for 78% of total user fees. 

A 1998 Supreme Court of Canada decision 
concluded that user fees could be considered 



unlawful and therefore may be repayable if they 
were determined by a court to be a tax that had not 
been established by enacted legislation or if the 
fee amounts charged were excessive and did not 
have a reasonable relationship to the cost of the 
services provided. Although the Ontario govern- 
ment has taken some action over the past decade to 
address this ruling, at the time of our audit in 2009 
there were still user-fee revenues collected by the 
Alcohol and Gaming Commission and the Ministry 
of Government Services of more than $500 million 
annually that may be at risk because they may not 
have met the Supreme Court's criteria for valid fees. 

The Non-Tax Revenue Directive, established in 
1991, is intended to maximize the Ontario govern- 
ment's non-tax revenues, including user fees, and 
to ensure that ministries regularly review services 
and rates, and keep non-tax revenue rates up to 
date. However, we noted in our 2009 Annual Report 
that the existing processes were, for the most part, 
not effective in achieving the Directive's goals. In 
addition, unlike user-fee legislation in place feder- 
ally and in some other provinces, Ontario's existing 
policies and procedures lacked transparency and 
public involvement in key decisions about changes 
to user-fee rates, nor was there sufficient public 
reporting on fees collected, their use, and the costs 
associated with providing the fee-related services. 

A key principle of the Directive is that, when 
it is reasonable and practical to do so, the cost of 
providing services to the public should be borne 
by those who benefit from the service. In 2008, as 
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Figure 1: Revenues from User Fees, 2010/11 ($ million) 

Source of data: Public Accounts of Ontario 



Total $1,976.6 



other - various ministries and agencies ($544.0) 



company registrations - Ministry of Government Services ($24.5) . 

gaming registrations and lottery event licences • 
Alcohol and Gaming Commission ($21.9) 

Drive Clean program - Ministry of the Enviornment ($33.0) 



personal property registrations - Ministry of Government Services ($38.5) 
estate and court-related fees - Ministry of the Attorney General ($53.6) 

liquor licences and permits - Alcohol and Gaming Commission ($180.7) 




driver's and carrier's licences and vehicle registrations - Ministry of Transportation ($1,080.4) 



part of the Budget process, the Ministry of Finance 
conducted a one-time review, which indicated that 
forecast user-fee revenues would recover less than 
75% of the costs identified for these fee-related 
services. In cases where a ministry decides not to 
charge the full cost of a service — such as when it is 
not practical or economical to do so, or users can- 
not afford to pay — the Directive requires that the 
ministry document the reasons for setting fees at 
reduced rates. We noted in 2009 that, for the most 
part, this was not being done. 

In addition, there were generally no recurring 
processes in place to keep user-fee rates up to date, 
as is required under the Directive. We noted many 
examples of fees with no rate-increase for 10 to 20 
years, despite the fact that the fees recovered only 
from 23% to 45% of the full costs of providing the 
services. 

Ministry of Finance guidelines require that 
ministries discount fees for services provided elec- 
tronically, to encourage their increased use by the 
public. We noted that no discounts were offered for 
driver's licences and vehicle registrations via the 
Internet or at electronic kiosks. On the contrary, 
services at electronic kiosks across the province 



incur a so-called "convenience" surcharge of one 
dollar per transaction. 

We made a number of recommendations for 
improvement and received commitments from the 
ministries that they would take action to address 
our concerns. 



Status of Actions Taken on 
Recommendations 



On the basis of information we received from the 
Ministry of Finance, which also co-ordinated addi- 
tional responses from other ministries, we noted 
that some progress has been made to address our 
recommendations. For instance, legislation was 
passed to replace the $470 million in annual beer 
and wine/ees with beer and wine taxes effective 
July 2010 in order to provide legislative clarity 
with respect to these revenues. As well, these taxes 
are being separately disclosed in the Province's 
2010/11 financial statements. The Ministry of 
Finance has also obtained a commitment from each 
ministry to review all existing non-tax revenues 
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over the next several years as part of the Results- 
based Planning process. However, the implementa- 
tion of several of our recommendations will be 
incumbent on a multi-ministry working committee 
that is expected to make recommendations by the 
end of the 2011/12 fiscal year for changes to the 
Ministry of Finance's policies and to the Non-Tax 
Revenue Directive. 

The status of action taken on each of our recom- 
mendations is as follows. 

POLICY AND CONTROL FRAMEWORK 
OVER USER FEES 

User Fees versus Taxes 

Recommendation 1 

To ensure that user fee revenues are not at risk of 
repayment because they are unconstitutional, the 
Ministry of Finance should obtain the legal assurances 
it needs or consider legislated or other changes that 
would protect the validity of these revenues. 

Status 

As part of the Ontario Tax Plan for More Jobs and 
Growth Act, 2009 (the 2010 Ontario Budget), new 
beer and wine taxes were imposed to replace the 
combination of beer and wine fees and revenue 
that would have been lost from lowering the sales- 
tax rate on alcohol associated with these fees. 
Effective July 1, 2010, approximately $470 million 
in manufacturers' fees that were previously paid 
to the Alcohol and Gaming Commission of Ontario 
by breweries, microbreweries, and wineries were 
replaced with beer and wine taxes collected by 
manufacturers from consumers and payable to the 
Ministry of Revenue. According to the 2010 Ontario 
Budget, this change is revenue-neutral for the 
province. 

Our 2009 Annual Report also noted that the 
Ministry of Government Services was collecting 
revenues for certain registration services that were 
at risk of constitutional challenge because the rev- 
enues exceeded the cost of providing the services 
by approximately $60 million, and it did not have 



an action plan to address this risk. The Ministry of 
Government Services has since completed a costing 
and pricing review of its fees as part of the 2011/12 
Results-based Planning process and has identified 
potential remediation strategies, including the 
possible development of a plan to reduce the fees 
over time. We were informed that the Ministry of 
Government Services is currently working with the 
Ministry of Finance to develop a strategy to address 
this issue for consideration by the Treasury Board/ 
Management Board of Cabinet; however, no time- 
table was provided for completing this. 

Policy Framework and Processes 

Recommendation 2 

To improve accountability, openness, and transpar- 
ency in decisions related to user fees and compliance 
with policies, the Ministry of Finance should research 
legislation, policies, and processes in use or planned 
in other jurisdictions to identify best practices that 
could be applied in Ontario. It should also consider 
making available to the Legislature and the public, 
as some other provinces do, information on decisions 
related to user fees, such as the extent to which fees 
are expected to recover costs, and requirements for 
proposing new fees and fee increases. 

Status 

A Non-Tax Revenue Working Group was established 
in July 2010 to review the government's approach 
to non-tax revenue. The Working Group is focusing 
on aligning the existing Costing and Pricing Policy 
guidelines and the Non-Tax Revenue Directive with 
public policy choices while ensuring compliance 
with applicable case law. The group is made up 
of representatives from the Ministry of Finance 
and several ministries that collect a significant 
amount of non-tax revenues, such as the Ministries 
of Transportation, Government Services, and the 
Environment. In addition to being given a mandate 
to review the recommendations we made in our 
2009 Annual Report, by the end of the 2011/12 fis- 
cal year the Working Group was to present options 
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to senior decision-makers for proposed revisions to 
the Ministry of Finance's Costing and Pricing Policy 
and Guidelines and the Non-Tax Revenue Directive. 

In November 2010, the Ministry of Finance com- 
pleted a jurisdictional review of non-tax revenue 
best practices, including consideration of areas 
such as legislation, policy, cost recovery, transpar- 
ency, reporting, and indexation. Detailed responses 
were received from six provinces, one territory, 
and the federal government. The review noted 
that the type of information jurisdictions make 
available to the public regarding decisions related 
to user fees varies between jurisdictions. We were 
advised that the Working Group will evaluate the 
responses and provide recommendations at the end 
of the 2011/12 fiscal year on practices that could be 
applicable in Ontario, including practices related 
to public reporting and transparency, subject to the 
approval of Treasury Board/Management Board of 
Cabinet. 

FEE PRICING AND COSTS 

Cost Recovery for Services 

Recommendation 3 

To meet the intent of the Non-Tax Revenue Directive 
that non-tax revenues be maximized, user-fee rates 
should be set at levels that would recover the costs of 
providing services where it is reasonable and practical 
to do so. Where full costs are not being recovered, there 
should be adequate documented rationale. As well, 
the Ministry of Finance, in conjunction with the other 
ministries and with Treasury Board approval, should 
consider establishing target cost-recovery ratios for 
services for which full costs are not being recovered. 

Status 

The Ministry of Finance advised us that the annual 
Results-based Planning process has been changed 
to require that business cases prepared by ministries 
proposing new non-tax revenues or changes to 
existing fees include information on cost recovery. 
While the Ministry of Finance does not require 
ministries to set target cost-recovery ratios for ser- 



vices provided at fees below full cost, ministries are 
required to indicate whether a proposed fee is below 
full cost recovery and, if so, to explain why, and to 
calculate the percentage of full cost recovery for any 
proposed change to a fee or any new fee. 

We were informed that, as part of the 2011/12 
Results-based Planning process, the Ministry 
of Finance required ministries to commit to 
developing a multi-year plan to review all existing 
non-tax revenue sources to ensure consistency with 
government policies and guidelines. Ministries are 
required to develop their plans over the next several 
years with scopes and timelines that best suit their 
individual circumstances, although the Ministry of 
Finance has imposed no deadline for ministries to 
complete these plans. We were also advised that, 
at the time of our follow-up, the Working Group 
was working with ministries to determine the most 
appropriate cost-recovery options. 

Updating Fee Amounts 

Recommendation 4 

To help ensure that ministries comply with existing 
policies requiring them to keep fee rates up to date 
with costs being incurred, the Ministry of Finance 
should work with ministries to establish regular 
processes for identifying changes in the costs of service 
delivery and for making formal recommendations to 
the Treasury Board for regularly updating fee rates. 

Status 

The Ministry advised us that a multi-year review 
of all non-tax revenues, which the ministries have 
committed to as part of their future annual Results- 
based Planning processes, will identify fees set at 
rates below full cost recovery and will state the 
impact of any changes to the rates. 

As part of its study of best practices across juris- 
dictions, the Working Group has identified regular 
review processes used in other jurisdictions, such 
as annual reviews and regular updates to fees based 
on indices such as the Consumer Price Index. The 
Working Group was in the process of evaluating 



Government User Fees 




these practices and the feasibility of implementing 
them in Ontario and will consider them for its 
recommendations when it submits the results of its 
review by the end of 2011/12. 

The Ministry advised us that the Treasury Board 
and the Management Board of Cabinet considers 
changes to non-tax revenue fees on a case-by-case 
basis during the annual Results-based Planning 
processes. Decisions on individual fees are made 
using a documented business case within the con- 
text of applicable case law as well as the existing 
Costing and Pricing Policy guidelines and public 
policy choices. 

Fees for Electronic Service Delivery 

Recommendation 5 

The Ministry of Transportation, in conjunction with 
the Ministry of Government Services, should compare 
its costs for delivering services via electronic kiosk and 
online with those of over-the-counter, in-person service 
delivery to establish whether "convenience" fees added 
to electronic kiosk services are justified and whether 
kiosk and online service delivery should be discounted. 

Status 

In December 2010, the Ministry of Government Ser- 
vices completed a product and service costing and 
fee review of services delivered by ServiceOntario, 
and submitted the results as part of the 2011/12 
Results-based Planning process. The review included 
an analysis of ServiceOntario's costs for those pro- 
grams where it collects fees on behalf of partner min- 
istries, such as the Ministry of Transportation (such 
as those for driver licensing and vehicle registration). 
While the results of the review were shared with the 
partner ministries, we were informed that work has 
yet to be done on integrating ServiceOntario's costs 
with those of the partner ministries to form a basis 
of comparison of costs between alternative delivery 
methods (e.g., electronic versus in-person), and 
for determining whether the "convenience" fees of 
approximately $749,000 in 2010/11 ($849,000 in 
2008/09) collected at kiosks were justified. We were 



advised that the Ministry of Government Services 
will be reviewing the kiosk fee as part of the kiosk 
strategy being developed for the expiry of the cur- 
rent contract in 2013. 

Enforcement and Compliance Costs 

Recommendation 6 

To ensure that accurate and consistent information is 
available for making informed decisions on fee rates, 
the Ministry of Finance should amend its Costing 
and Pricing Policy and guidelines used by ministries 
to require that compliance and enforcement costs be 
appropriately considered when determining the full 
cost of fee-related services. 

Status 

We were advised that the mandate of the Working 
Group includes a review of the Costing and Pricing 
Policy and guidelines used by ministries, including 
whether compliance and enforcement costs will be 
used in arriving at the full cost of fee-related servi- 
ces. The results are expected by the end of 2011/12. 

REVENUE COLLECTION 

Recommendation 7 

The Ministry of the Environment should obtain 
periodic internal or external audit and other assur- 
ances that the revenues collected and remitted by the 
private-sector operators of its Drive Clean program 
are accurate. 

Status 

The Ministry of the Environment requested the 
Ontario Internal Audit Division to complete an 
audit of the accuracy of revenue collected for the 
Drive Clean program. For the period from Nov- 
ember 2008 to November 2009, it examined the 
accuracy of revenues collected and remitted by 
the private-sector operators of the program, and 
assessed the Drive Clean Office's processes and its 
private-sector service provider's revenue manage- 
ment controls that assure the accuracy of revenues 
collected. This included emission-test recording 
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and verification processes, and billing and collec- 
tion processes for revenues from approved facilities. 
In their July 2010 report, the internal auditors con- 
cluded that, overall, effective controls were in place 
to reduce the risk of revenue loss to an acceptable 
level, and they made several recommendations for 
improving controls. We were advised that another 
internal audit of the program's revenue collection 
process has been scheduled for 2011/12. 

The Ministry of the Environment entered into a 
new contract with the private-sector operator of the 
Drive Clean program in January 2011. The contract 
requires the operator to have an independent exter- 
nal auditor conduct an annual audit following a 
process approved by the province to ensure that all 
information systems and applications are operating 
in accordance with provincially-approved specifica- 
tions and that all operating procedures conform 
to those that were approved by the province. The 
Ministry of the Environment expects to receive the 
first external audit report in January 2013. 

The Ministry of the Environment also reconciles 
Drive Clean tests recorded on the Ministry of Trans- 
portation vehicle database with revenues from the 
program. 

SERVICE STANDARDS AND REPORTING 

Recommendation 8 

To enhance accountability and reporting over min- 
istries' fee-related services, the Ministry of Finance, 
in conjunction with ministries, should identify and 



implement the best practices in use in other jurisdic- 
tions relating to establishing and publicly reporting 
service standards and actual service levels achieved. 

Status 

In our 2009 Annual Report, we noted that certain 
other jurisdictions impose requirements for depart- 
ments, agencies, boards, and commissions to 
report on how their standards compare to those 
established by other countries with which a rel- 
evant comparison can be made and against which 
performance can be measured. In addition, clients 
must be provided with explanations of how user 
fees are determined and of their related costs and 
revenues. In this way, clients can clearly see the 
costs of the services they pay for in relation to what 
they receive. 

The Ministry of Government Services indicated 
that it has worked with ministries in the Ontario 
Public Service to assist with the implementation of 
the new Service Directive that came into effect in 
January 2010. This Directive requires ministries to 
establish program-specific standards for services 
offered, for monitoring and measuring the quality 
of services provided, and for communicating to cus- 
tomers the actual level of service achieved. We were 
informed that the Ministry of Government Services 
has been working with the Working Group to assess 
best practices in other jurisdictions, including prac- 
tices relating to public reporting, and was to cover 
this area in its recommendations expected at the 
end of 2011/12. 
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Background 




Long-term-care nursing homes and homes for 
the aged (now collectively called long-term-care 
homes) provide care, services, and accommoda- 
tions to individuals unable to live independently 
and requiring the availability of 24-hour care. 
There are more than 600 such homes in Ontario, 
caring for about 75,000 residents, most of whom 
are over 65 years old. In the 2010/11 fiscal year, 
funding to long-term-care homes by the Ministry of 
Health and Long-Term Care (Ministry) through the 
Local Health Integration Networks totalled $3 bil- 
lion ($2.8 billion in the 2008/09 fiscal year), with 
residents generally also making a co-payment to the 
home of between $1,600 and $2,200 per month. 

There is a high risk of infectious organisms/ 
diseases — such as Clostridium difficile (C. difficile), 
febrile respiratory illness (FRI) (e.g., colds, influ- 
enza, pneumonia), methicillin-resistant Staphylo- 
coccus aureus (MRS A), and vancomycin-resistant 
enterococci (VRE) — spreading among residents 
of long-term-care homes because they often share 
rooms and generally eat and participate in activities 
together. As well, older residents are generally more 
vulnerable to illness. When a resident acquires an 



infection in a home, it is considered a health-care- 
associated infection (HAI) . Numerous parties play 
a role in preventing and controlling the spread of 
infections in long-term-care homes, as shown in 
Figure 1. 

In 2008, we conducted an audit of infection 
prevention and control in hospitals. In 2009, we 
used the knowledge gained on that audit to conduct 
a similar audit in the long-term-care home environ- 
ment. We found that all three long-term-care 
homes we visited — Extendicare York in Sudbury, 
Nisbet Lodge in Toronto, and Regency Manor in 
Port Hope — had a number of processes in place 
to prevent and control HAIs. Furthermore, these 
homes had all recently conducted their first review 
of staff compliance with certain hand-hygiene 
policies, since hand hygiene is the most important 
activity for controlling the spread of infections. 
However, we noted areas where these homes could 
improve their practices. Some of our more signifi- 
cant observations included the following: 

• The Ministry did not have information on the 
total number of cases of HAIs in long-term- 
care homes. The information collected at the 
homes we visited was generally not compar- 
able because the homes defined and counted 
HAIs in different ways. 
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Figure 1: Selected Key Roles and Responsibilities for Infection Prevention and Control in Long-term-care Homes 

Prepared by the Office of the Auditor General of Ontario 
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• Although the homes visited had policies to 
screen new residents for FRIs, documentation 
at two of the homes indicated that just 60% to 
80% of new residents sampled were screened. 
At the third home, there was no evidence of 
formal screening for FRIs. 

• Each home had a policy to test new residents 
for tuberculosis within 14 days of admission, 
as required by legislation. One home tested all 
new residents in our sample, but the other two 
tested only 70% and 80%, and often much 
later than within the required 14 days. 

• Homes generally did not have unoccupied 
rooms to move infectious residents into. 
Although Ontario's Provincial Infectious 
Diseases Advisory Committee (PIDAC) indi- 
cates that residents with an FRI who share a 
room should have the curtain drawn around 
their bed, all three homes indicated that they 
would pull a curtain around a resident's bed 
only if the resident requested it. 

• Although PIDAC recommends cleaning the 
rooms of residents who have C. difficile twice a 
day, none of the homes did this. 

• In the 2008/09 fiscal year, 81 C. difficile 
outbreaks in homes were reported to the 
Ministry. The increased use of antibiotics has 
been shown to increase the risk of C. difficile. 
None of the homes had a formulary that lists 
the antibiotics that physicians can prescribe, 
as recommended by PIDAC. 

• Unlike hospitals, long-term-care homes are 
not required to report publicly on certain 
patient-safety indicators, such as health-care- 
acquired cases of C. difficile, MRSA, and VRE, 
as well as hand-hygiene compliance among 
health-care workers. 

• None of the Infection Prevention and Control 
Professionals designated by the homes had 
taken the specific training recommended by 
PIDAC. 



STANDING COMMITTEE ON 
PUBLIC ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this audit in May 2010. In Febru- 
ary 2011, the Committee tabled a report in the 
Legislature resulting from this hearing. The report 
contained 11 recommendations and requested that 
the Ministry report back to the Committee with 
respect to the following issues: 

• how best to ensure that hospitals and long- 
term-care homes exchange information on 
patients with infectious diseases who transfer 
to or from a long-term-care home; 

• the steps long-term-care homes would be 
taking to implement cohorting or isolating 
patients suspected of having an HAI, espe- 
cially given the limited availability of private 
rooms; 

• whether the Ministry would be requiring each 
long-term-care home to publicly report the 
influenza immunization rates for its residents 
and staff, and whether the Ministry would 
also be publicly reporting this information for 
each long-term-care home; 

• the Ministry's assessment of the advantages 
and disadvantages of mandatory influenza 
immunization programs for staff of long-term- 
care homes; 

• the steps taken to periodically review the use 
of antibiotics in each long-term-care home; 

• whether homes that had not yet undertaken 
a Medication Safety Self -Assessment would 
be required to do so, and, if so, by what date, 
as well as how frequently the self-assessment 
would be required to be completed; 

• whether the Ministry would be establishing 
benchmark standards for infection rates in 
long-term-care homes by type of infection, 
and whether long-term-care homes would be 
required to publicly report comparable infor- 
mation on infection rates; and 
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• the number and percentage of long-term-care 
homes that then had a trained and certified 
infection-prevention-and-control professional 
on staff and, if a home did have one, how 
infection-prevention-and-control expertise 
could be accessed if needed. 
The Ministry formally responded to the Com- 
mittee in July 2011. A number of the issues raised 
by the Committee were similar to our observations. 
Where the Committee's recommendations are 
similar to ours, this follow-up includes the recent 
actions reported by the Ministry to address the con- 
cerns raised by both the Committee and our 2009 
audit. 



Status of Actions Taken o 
Recommendations 



The long-term-care homes and the Ministry 
provided us with information in spring and sum- 
mer 2011 on whether and the extent to which they 
had implemented our recommendations. According 
to this information, some progress has been made 
in implementing many of the recommendations we 
made in our 2009 Annual Report. The current status 
of the actions taken by the Ministry and the homes 
is summarized following each recommendation. 

SCREENING 

Recommendation 1 

To ensure that residents with infectious diseases are 
identified quickly enough to minimize the risk of the 
disease spreading to others, long-term-care homes 
should periodically monitor whether their screening 
processes are in accordance with the recommenda- 
tions made by the Provincial Infectious Diseases 
Advisory Committee and legislative requirements. 

Status 

At the time of our follow-up, all three of the audited 
long-term-care homes indicated that they were 



monitoring whether their screening policies are in 
accordance with the recommendations made by the 
Provincial Infectious Diseases Advisory Committee 
(PIDAC) and legislative requirements. They noted 
that they either had updated or were in the process 
of updating these policies. One of the homes com- 
mented that it receives regular notifications of 
PIDAC updates from its Regional Infection Control 
Network (RICN), which the home uses to update its 
practices. Another home commented that its Infec- 
tion Prevention and Control Committee analyzes its 
screening processes on a regular basis. This home 
also noted that it was continuing to work with its 
local hospital to get the hospital's screening results 
for patients who will be coming or returning to the 
home as residents. The third home indicated that it 
had reviewed compliance with its MRSA and VRE 
admission and readmission screening protocols 
for 2009 to 2011 and noted that compliance had 
improved after the protocols were reinforced with 
staff. The Ministry indicated that all of the long- 
term-care homes in Ontario have now implemented 
a new computerized assessment-and-planning 
system as part of a ministry initiative. The Ministry 
noted that this system can help prevent and control 
infections by having homes monitor and report 
symptoms more consistently. 

RESIDENT CARE 

Recommendation 2 

In order to better prevent the transmission of infec- 
tious diseases: 

• long-term-care homes should monitor whether 
prevention best practices (such as hand hygiene 
and the use of personal protective equipment) 
and infection-specific precautions (such as 
twice-daily cleaning of rooms of residents who 
have C. difficile} are conducted in accordance 
with the recommendations made by the Prov- 
incial Infectious Diseases Advisory Committee 
(PIDAC) and review their monitoring methodol- 
ogy to ensure that abnormally high compliance 
rates are reflective of actual practices; 
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• the Ministry of Health and Long-Term Care 
should develop guidance to assist homes in 
determining how best to meet PIDAC's recom- 
mendations on isolating and cohorting residents 
who have or are at high risk of having infectious 
diseases, given the limited availability of private 
rooms; and 

• long-term-care homes should continue to 
promote and monitor the immunization of resi- 
dents and staff. 

To help prevent residents from acquiring an 
infected skin breakdown, long-term-care homes 
should adopt processes, such as using a sign-off sheet 
for recording when residents are repositioned, to 
enable supervisory staff to monitor compliance with 
established procedures. 

Status 

At the time of our follow-up, the three audited 
homes indicated that they were all monitoring their 
hand-hygiene initiatives. They noted that they had 
implemented the "Just Clean Your Hands" program, 
which included staff education and hand-hygiene 
audits. All three homes also noted that they were 
providing immediate feedback to staff persons 
not complying with hand-hygiene protocols. As 
well, two of the homes were monitoring the use of 
personal protective equipment, and the third home 
indicated that it planned to commence monitoring 
in late 2011. The three homes also indicated that 
twice-daily cleaning of rooms of residents who have 
C. difficile was now being conducted in accordance 
with PIDAC recommendations. Two of the homes 
commented that they used a checklist to ensure 
compliance, and the third home indicated that it 
planned to implement, by fall 2011, a checklist 
for the twice-daily cleaning of the bathrooms of 
residents with C. difficile. One of these homes noted 
that its manager was reviewing and signing com- 
pleted housekeeping checklists and that the check- 
lists were also being used to direct room cleaning 
during outbreaks to minimize the spread and sever- 
ity of the outbreak. Another home noted that it was 
providing a commode for any resident with C. dif- 



ficile who shared a room so that this resident would 
not share the bathroom with other residents. 

The Ministry told us that although it had not 
issued any specific guidance to homes on isolating 
or cohorting residents who are at high risk of hav- 
ing infectious diseases, it expected that new room 
designs (part of its Renewal Strategy, announced in 
2007, involving 35,000 older long-term-care home 
beds) would assist homes in following PIDAC's rec- 
ommendations. The Ministry further indicated that 
the strategy's first phase, involving almost 4,100 
long-term-care home beds, would be completed by 
2015, with the remaining phases being completed 
within 10 to 15 years. The Ministry also noted that, 
in the case of the local Public Health Unit declaring 
an outbreak at a home, ministry inspectors work 
collaboratively with the Unit to identify strategies 
and best practices to manage the outbreak. 

All three long-term-care homes indicated that 
they were continuing to promote and monitor 
the immunization of residents and staff. One 
home stated that its seasonal influenza campaign 
included education as well as immunization. 
Another home commented that it ran influenza 
clinics for residents, staff, volunteers, and visitors, 
and provided incentives to staff to get the annual 
influenza shot. The third home noted that it pro- 
moted immunization by displaying posters and 
other promotional material provided by its local 
Public Health Unit and Regional Infection Control 
Network, as well as by providing information to 
both residents and staff. However, this home com- 
mented that its 2011 influenza immunization rates 
were below expectations. It believed that this was 
the result of misunderstandings about the H1N1 
immunization vaccination. The Ministry high- 
lighted that a regulation under the Long-Term Care 
Homes Act, 2007, which came into force on July 1, 
2010, specifically requires long-term-care homes to 
have an immunization program for both residents 
and staff. 

As for preventing infected skin breakdowns, 
one home indicated that in October 2010 it imple- 
mented a sign-off sheet to record when residents 
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are repositioned. The other two homes indicated 
that they had electronic point-of-care systems (that 
is, patient charting systems that can be used in 
the same location where the resident's care is pro- 
vided). One of these homes noted that its system 
enabled it to create resident-specific monitoring 
when necessary. This home also indicated that 
senior staff were informally monitoring resident 
repositioning when observing the care practices 
of their staff. The other home noted that, for 
residents requiring frequent repositioning, staff 
were documenting the repositioning on the home's 
electronic point-of-care system. The home indicated 
that it was also monitoring residents with skin 
breakdowns and analyzing data relating to newly 
acquired and worsening skin breakdowns. Further- 
more, to help prevent residents from acquiring an 
infected skin breakdown, the home's wound and 
skin co-ordinator was assessing resident skin break- 
downs weekly and, where needed, also reviewing 
resident treatment plans, educating staff, and liais- 
ing with physicians. 

ANTIBIOTIC USE 

Recommendation 3 

To help prevent antibiotic-resistant organisms and 
reduce the susceptibility of residents to certain infec- 
tions, the Ministry of Health and Long-Term Care, 
in conjunction with other interested stakeholders, 
should: 

• assist long-term-care homes to develop a drug 
formulary; and 

• periodically review the use of antibiotics in 
long-term-care homes so that follow-up action 
can be taken where the use of antibiotics seems 
unusually high. 

Status 

The Ministry noted in its follow-up response 
regarding this recommendation that it was con- 
tinuing to rely on physicians to determine which 
antibiotics to prescribe to residents and on pharma- 
cies to regularly review the use of antibiotics at 



long-term-care homes. The Ministry also stated that 
a regulation under the Long-Term Care Homes Act, 
2007 that came into effect July 1, 2010, requires 
that every long-term-care home develop an inter- 
disciplinary medication management system. The 
system is to provide for appropriate and safe use 
of medications, including antibiotics, as well as 
effective and optimal drug therapy outcomes for 
residents. 

The regulation further requires that each long- 
term-care home establish a multidisciplinary team 
to review drug utilization trends and patterns quar- 
terly and take action where necessary. The multi- 
disciplinary team is to include the home's medical 
director (who is often the physician prescribing 
medications for many of the home's residents) 
as well as the home's administrator, the home's 
director of care, and in most cases a pharmacist. 
The Ministry indicated that it expected the review 
of drug utilization trends and patterns to include a 
review of antibiotic usage. The Ministry also noted 
that long-term-care homes can reduce the need for 
antibiotics by encouraging residents to be immun- 
ized against specific infectious diseases such as 
pneumococcal pneumonia, tetanus, and diphtheria. 

At the time of our follow-up, all three of the 
audited homes told us that they were reviewing 
antibiotic usage quarterly. 

SURVEILLANCE 

Recommendation 4 

To enhance the effectiveness ofinfection-prevention- 
and-control programs, the Ministry of Health and 
Long-Term Care, in conjunction with the long-term- 
care homes, should: 

• require long-term-care homes to identify and 
track infections in a consistent and comparable 
manner, using standard definitions and surveil- 
lance methods; 

• establish reasonable targeted maximum rates/ 
benchmarks for the more prevalent infections; 
and 
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• look into requiring that long-term-care homes 
report publicly, as hospitals do, on certain 
patient-safety indicators, such as cases ofC. dif- 
ficile and hand-hygiene compliance among 
resident-care staff, using standard definitions 
and surveillance methods. 
As well, long-term-care homes should ensure that 
staff, including designated infection-prevention-and- 
control professionals, have the infection-surveillance 
training recommended for their position. 

Status 

The Long-Term Care Homes Act, 2007, which came 
into force on July I, 2010, requires that long-term- 
care homes establish an infection-prevention-and- 
control program. This includes daily monitoring 
to detect infections. In addition, the regulation 
under the Act requires that symptoms indicat- 
ing the presence of an infection be monitored in 
accordance with evidence-based practices where 
they exist and, if there are none, in accordance with 
prevailing practices. The regulation also requires 
that symptoms be analyzed daily and reviewed at 
least monthly to detect trends, for the purpose of 
reducing the incidence of infection and outbreaks. 
The Ministry commented that each home decides 
how best to identify and track infections, rather 
than there being a standard definition of an infec- 
tion and standard surveillance methods. Neverthe- 
less, the Ministry was working toward developing 
standard definitions and surveillance methods. 
One of the homes indicated that it was using a 
spreadsheet to track infection data from each unit 
within the home. The home was using the results 
to develop an action plan to reduce the number of 
infections in the home. 

At the time of our follow-up, the Ministry 
indicated that it encourages long-term-care homes 
to follow basic surveillance principles, includ- 
ing determining rates of respiratory and enteric 
(gastrointestinal-tract) infections (generally called 
the baseline rate) that can be compared to future 
rates of these infections. The Ministry stated that 
this would enable the long-term-care home to 



assess the impact of its infection-prevention-and- 
control program over time. The Ministry further 
commented that this was a more appropriate 
approach than using system-wide benchmarks inso- 
far as a home's rates of certain infections, such as 
influenza, tend to be more influenced by the pres- 
ence of the infection in the local community than 
by the home's infection-prevention-and-control 
practices. 

The Ministry indicated that it had examined 
whether long-term-care homes should be required 
to publicly report patient-safety indicators such as 
HAI rates, as hospitals do. The Ministry noted that 
it had consulted with the long-term-care homes 
and other stakeholders and that there was a high 
level of satisfaction with the current extent of 
voluntary public reporting through Health Quality 
Ontario. Although Health Quality Ontario does not 
provide public information on cases of C. difficile 
or hand-hygiene compliance among resident-care 
staff, it does report on other patient-safety indica- 
tors, such as the percentage of residents with 
worsening bladder function and the percentage 
of residents who had a new pressure ulcer (such 
as a bedsore) or a pressure ulcer that recently got 
worse. At the time of our follow-up, only about 125 
long-term-care homes, including the three homes 
audited, reported information publicly on the 
Health Quality Ontario site. However, the Ministry 
anticipated that all homes would be participating 
by March 2012. The Ministry also indicated that it 
would re-evaluate in the future the decision not to 
have homes publicly report on C. difficile and hand- 
hygiene compliance among resident-care staff. 

One home indicated that its infection-control 
practitioner has attended several infection-control 
and quality-management workshops held by its 
Regional Infection Control Network and its Local 
Health Integration Network. As well, the infection- 
control practitioner had applied to take a non- 
acute-care infection-control practitioners' course 
offered by its Regional Infection Control Network 
but did not get accepted due to the overwhelming 
response to the course. Furthermore, this home 
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noted that infection-control training is provided 
to all staff in accordance with the requirements of 
their position. This includes education on routine 
practices and special precautions, cleaning and 
disinfection, and hand hygiene. 

Another home indicated that its infection- 
control practitioner was scheduled for a formal 
surveillance training course at a college in fall 2011. 
This home also told us that its staff completed 
general infection-control training in 2010, which 
included training on hand hygiene and the proper 
use of personal protective equipment. This home 
further noted that infection-control training was 
done again in May 2011. 

The third home indicated that it was recruiting 
a new Assistant Director of Care who would be the 
home's infection-control practitioner. If the person 
hired does not have sufficient training in infection 
prevention and control, the home will arrange for 
such training. In the interim, the home has access 
to the expertise of professionals employed by the 
home's corporate owner, including a person certi- 
fied in infection control. 



Chapter 4 



Ministry of Education 

Section 

Literacy and Numeracy 
Secretariat 

Follow-up on VFM Section 3.07, 2009 Annual Report 



Background 



The Ministry of Education (Ministry) is responsible 
for the system of publicly funded elementary 
and secondary school education in Ontario. Its 
responsibilities include developing the primary and 
secondary school curricula, setting requirements 
for student diplomas, and providing funding 
to school boards. The Ministry also set up the 
Education Quality and Accountability Office 
(EQAO) — a government agency — to provide 
independent assessments of student achieve- 
ment by testing students in reading, writing, 
and mathematics. The Ministry's Literacy and 
Numeracy Secretariat (Secretariat), the subject 
of one of our 2009 audits and this follow-up, was 
established in November 2004 to help more than 
4,000 elementary schools across 72 school boards 
to meet student-achievement targets. From the 
time it was established in 2004 to March 31, 2011, 
the Secretariat spent $505 million ($340 million 
by March 31, 2009), with almost $423 million 
($288 million by March 31, 2009) transferred to 
school boards. 

The Ontario government made a significant 
commitment to improving student achievement 
when, in 2004, it set a goal that 75% of all 12-year- 
olds (grade 6 students) would score a level-three 



standard (approximately a B average) on province- 
wide testing for reading, writing, and mathematics 
by 2008. Although the Ministry had not achieved 
this goal by 2008, substantial progress had been 
made over the five years previous to our 2009 audit, 
and the percentage of 12-year-olds achieving the 
provincial standard increased from an average of 
56% in 2003/04 to 65% in 2007/08. We stated at 
the time of our audit in 2009 that further increasing 
this percentage would be a challenging undertaking 
and noted a number of improvements that could be 
made to help achieve this goal. Some of our more 
significant observations at the time of the 2009 
audit were: 

• Although the Secretariat and the school 
boards we visited had done some limited 
assessment of the effectiveness of the secre- 
tariat programs, further analysis was required 
if the Secretariat was to ensure that its fund- 
ing of almost $288 million had been directed 
to the initiatives that provide the most benefit. 

• School board improvement plans had been 
initiated to help teachers, principals, and 
school board staff plan and implement 
strategies to improve student achievement. 
The Ministry had developed a framework for 
an effective improvement-planning process. 
However, neither the Secretariat nor the 
boards we visited documented, monitored, or 
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reported on the plans to the extent necessary 
to assess whether the plans were contribut- 
ing to improved student achievement. Also, 
because it exercised only limited oversight, 
the Secretariat did not have the information 
needed to identify patterns and trends among 
school boards, so it could not identify the 
most successful initiatives and share them 
with other boards. 

• Secretariat program funding was not always 
allocated to school boards and schools with 
the greatest need. Rather, funding allocation 
was based on average daily enrolment or the 
reason a given amount of funding went to a 
school board could not be fully explained by 
the Secretariat. We found that, for one major 
program, the board with the greatest number 
of schools designated as low-performing 
received only $17 per student, while several 
boards with no schools designated as low- 
performing received more than twice this 
amount per student. 

• The Secretariat routinely uses certain boards 
as "bankers" to act as distributors of funds 

to third parties or other school boards. We 
questioned the need for such arrangements 
and noted that there is no memorandum of 
understanding or agreement between the 
Secretariat and the banker boards outlining 
respective roles and responsibilities, account- 
ability relationships, reporting requirements, 
and service levels to be provided. Also, the 
Secretariat paid banker boards administrative 
fees that in some cases appeared excessive. 
We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 

STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this audit in May 2010. In Decem- 



ber 2010 the Committee tabled a report in the 
Legislature resulting from this hearing. The report 
contained six recommendations and requested that 
the Ministry of Education report back to the Com- 
mittee with respect to the following: 

• what measures the Ministry is considering to 
make better use of cohort tracking to assess 
the progress of the same group of students 
over time; 

• the criteria the Ministry uses to assess school 
board improvement plans, including an 
update on the template that the Ministry has 
developed for the boards to use and how it 
plans to communicate best practices to all 
school boards; 

• the results of the Ministry's review of the 
effectiveness of the Secretariat's various pro- 
grams, specifying the review criteria used, the 
results of previous reviews, and any program 
changes that have resulted from the reviews; 

• the Ministry's most recent data on the amount 
of funding allocated to secretariat programs 
on the basis of needs and the amount based 
on enrolment, its criteria for assessing 
whether program money has been spent 
effectively, whether it has identified any 
program funding that should be redirected, 
and the percentage of total funding that was 
actually spent on the intended services; 

• an interim report from the Ministry on 
its review of lead or banker board use, 
specifying: 

• whether the Ministry is on track to com- 
plete its review by the end of the 2010/11 
fiscal year; 

• what initiatives formerly administered by 
lead/banker boards have been returned to 
the Ministry; 

• whether the Ministry will continue to use 
lead/banker boards and, if so, what criteria 
it will use to select them and to monitor 
their expenditures; and 

• what criteria the Ministry will use to deter- 
mine the appropriate levels of payment for 
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services provided by lead/banker boards 
and for reviewing their expenses; and 
• an update of the Ministry's assessment of data 
from the study it commissioned to compare 
the consistency of students' report card marks 
with their grade 3 and grade 6 EQAO achieve- 
ment results and whether the Ministry will 
consider ongoing tracking of the correlation. 
The Ministry responded to the Committee in 
March 2011. A number of the issues raised by the 
Committee were similar to our observations. Where 
the Committee's recommendations are similar to 
ours, this follow-up includes the recent actions 
reported by the Ministry to address the concerns 
raised by both the Committee and our 2009 audit. 



Status of Actions Taken on 
Recommendations 



According to information received from the 
Ministry substantial progress has been made on 
implementing all of the recommendations in our 
2009 Annual Report. For example, the Ministry 
reported the completion of thorough reviews of 
the board improvement-planning process, of all 
the Secretariat's program initiatives, and of the 
correlation between report card marks and EQAO 
results. These reviews resulted in, for example, the 
development of new criteria for the board planning 
process and the elimination, revision, or expansion 
of several programs. The Ministry also completed 
the implementation of the Ontario Statistical 
Neighbours System, a system with demographic, 
school, and student performance information used 
for program planning, and the Ministry is training 
all boards in how to use it. The status of the action 
taken on each of our recommendations at the time 
of our follow-up was as follows. 



MEASURING AND REPORTING ON 
PROGRAM EFFECTIVENESS 

Recommendation 1 

The Ministry of Education should develop more com- 
prehensive indicators for measuring and reporting on 
its effectiveness in improving student achievement. In 
addition to reporting the percentage of 12-year-olds 
who are at or above the provincial standard, it should 
also consider reporting changes in the gap between 
the top-performing and lower-performing student 
groups and schools, as well as how specific student 
cohorts perform over time while participating in the 
programs and initiatives intended to improve their 
performance. 

Status 

The Ministry informed us that it uses EQAO test 
results over time as one method for measuring 
improvements in student learning and achievement 
in grades 3 and 6. Cohort tracking now provides the 
Ministry with an additional way to assess student 
progress and gives some insight into the effective- 
ness of the Ministry's previous efforts to support 
students. Such tracking has also become one of the 
means to identify areas where performance could 
be improved, so that information can be developed 
about how to focus support on these identified 
areas of need. 

Now that the EQAO has begun to regularly track 
cohorts of students, the Ministry has determined 
that most cohorts have been improving in reading, 
writing, and mathematics. For example, of the 
120,000 students who attended school in English- 
language school boards and who wrote both the 
grade 3 assessment in 2006/07 and the grade 6 
assessment in 2009/10, 10% more met the standard 
in reading in grade 6 than in grade 3, 6% more 
met the standard in writing, but 8% fewer met the 
standard in mathematics. For the 5,700 students 
who attended school in French-language school 
boards and who wrote both assessments, the results 
for reading, writing, and mathematics improved by 
24%, 6%, and 20%, respectively. 



2011 Annual Report of the Office of the Auditor General of Ontario 



The Ministry informed us that cohort-tracking 
data will continue to be used going forward from 
2009/10 as a needs assessment and improvement 
tool for school boards. The Ministry noted that it 
uses EQAO test results over time for its planning 
purposes and for selecting priorities. The Ministry 
also tracks the proportion of students at each EQAO 
performance level, one through four (similar to 
letter grades) . Level three, equivalent to a B grade, 
is an indication that the student has demonstrated 
most of the required skills and is currently at the 
provincial standard. Cohort tracking also includes 
analysis of specific groups of students, such as 
males and females, English- and French-language 
learners, and students with special needs. It com- 
pares the results for specific groups such as these 
to overall student results to help inform its plans to 
reduce the gaps in achievement. 

The Ministry also indicated that it has intro- 
duced changes to the assessment, evaluation, and 
reporting of elementary student achievement by 
implementing a new fall progress report that allows 
teachers to give parents personalized feedback on 
each student's strengths and on steps required for 
improvement without grading students. 

SCHOOL BOARD IMPROVEMENT PLANS 

Recommendation 2 

To ensure that the improvement-planning process is 
sufficient to support boards, administrators, prin- 
cipals, and teachers in helping students to improve 
results and progress toward the provincial standard 
in achievement testing, the Ministry of Education 
should: 

• implement a formal improvement-plan review 
process to help ensure that all of the necessary 
components of an effective plan are included; 

• require that school boards post improvement 
plans online to enhance accountability and 
transparency; 

• consider adopting the practice followed in some 
other provinces of using a formal contract with 
school boards that would require school boards 



to periodically report their results in achieving 
the goals in their improvement plans; and 

• properly document the result of its monitoring 
efforts along with any required corrective action 
to be taken and any subsequent follow-up where 
plans are not complete. 

Status 

We were informed that a ministry working group 
put in place a formal improvement-plan review 
process that articulates the criteria for good board 
improvement plans and provides an assessment 
template for improvement planning. These criteria 
include: 

• assessment of student needs and 
achievements; 

• identification of SMART (specific, measur- 
able, attainable, results-oriented, time-bound) 
goals; 

• development of indicators to measure student 
achievement; 

• identification of required human and financial 
resources; 

• identification of professional learning needs 
for educators; 

• a process for data collection and analysis; 

• clearly defined responsibilities for plan imple- 
mentation and monitoring; and 

• comparison of planned results with actual 
outcomes. 

The Ministry's student achievement officers will 
use these criteria to assess whether board goals are 
appropriate and are currently being met, and will 
document their review of the school board plans 
and provide feedback to boards that includes infor- 
mation on best practices. 

The Ministry has also developed a refined ver- 
sion of its School Effectiveness Framework to be 
used in the board improvement planning process to 
provide guidance for assessing student and profes- 
sional learning needs. In addition, the Ministry 
has developed and provided boards with other 
resources, such as examples of SMART goals to help 
set targets and develop indicators for monitoring 
student achievement. 
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Although the Ministry advised us that it has 
not formally required boards to post improvement 
plans online, we were informed that all boards 
have done so. The Ministry also noted that it has 
not entered into any formal contracts with school 
boards to periodically report their results in achiev- 
ing the goals in their improvement plans. However, 
revisions to the Education Act pursuant to Bill 177, 
the Student Achievement and School Board Govern- 
ance Act, 2009, increased the responsibility of 
boards to report publicly regarding their plans to 
promote student achievement and well-being. 

The Ministry informed us that it has put 
additional monitoring processes in place, with 
staff making three annual visits to school boards 
regarding their improvement plans. The first visit in 
the fall helps develop the board improvement plan 
and ensure that the Ministry's criteria are included 
before the final plan is submitted to the Ministry. 
The second visit, in the middle of the school year, is 
used to monitor the plan against the SMART goals, 
indicators, and targets to identify areas that require 
corrective action. The final visit, at the end of the 
school year, is to document, review, and evaluate 
the impact on student achievement. 

MONITORING AND FUNDING OF 
PROGRAM INITIATIVES 

Recommendation 3 

To ensure that student achievement initiatives are 
effective and that limited resources are used appropri- 
ately, the Literacy and Numeracy Secretariat should: 
m formally evaluate how well all its program 
initiatives contribute to improving student 
achievement, and modify or eliminate the less 
effective initiatives; 

• ensure that its program funds are allocated to 
the areas of greatest need; 

• ensure that program funds are being spent for 
the intended purpose; 

• ensure that expenditures made by the Council 
of Ontario Directors of Education are appropri- 
ately approved and supported; and 



• reconsider pre-flowing funds to "banker" school 
boards. 

Status 

The Ministry noted that, in 2010, it formalized a 
three-year plan to assess the Secretariat programs 
and initiatives. This review was guided by a logic 
model designed to systematically assess whether 
a program contributed to student learning and 
achievement, by aligning program goals, inputs, 
outputs, activities, and performance measures. 
Overall, the Ministry indicated that it had per- 
formed an initial assessment of all its programs. As 
a result of this initial review, one initiative, Leader- 
to-Leader, has been eliminated. Several other 
initiatives have been revised or expanded since we 
completed our 2009 Annual Report. Both external 
and internal research has been used as input for 
the assessment of student achievement initiatives, 
including the following reviews: 

• In December 2010, an internal review was 
completed on the Ontario Focused Interven- 
tion Partnership (OFIP) program, which 
focuses on schools where fewer than 50% of 
the students met the provincial requirement 
of level three on EQAO testing. The 2009/10 
results for the 137 OFIP schools showed an 
overall average improvement of five percent- 
age points, and in 62 of these schools more 
than 50% of the students met the provincial 
standard. The study noted that the strongest 
indicator of success reported by the 62 suc- 
cessful schools was their self-assessment using 
the School Effectiveness Framework. The 
results of this study were shared with school 
boards. 

• An external study contracted by the Ministry 
and completed in November 2010, Collab- 
orative Inquiry and Learning in Mathematics 
(CILM), identified that CILM is an appropri- 
ate approach for teaching and learning 
math. Based on these findings, the Ministry 
expanded CILM to all boards. In addition, 
internal research done by the Ministry on the 



2011 Annual Report of the Office of the Auditor General of Ontario 



Student Work Study Teacher Initiative found 
a number of factors contributing to the suc- 
cess of students' work moving from level two 
to level three (the provincial standard) . The 
Ministry has shared these success factors with 
school boards. 
• In January 2011, another external study was 
completed on the Schools on the Move initia- 
tive to identify which factors contributed to 
student achievement in the 32 participating 
schools that faced challenging circumstances. 
In particular, it identified four factors contrib- 
uting to student success that could be trans- 
ferred to other schools: support for a common 
code of behaviour throughout the school; a 
feeling of responsibility among teachers for 
student learning; teachers' commitment to 
the notion that all students can learn suc- 
cessfully; and the explicit teaching of literacy 
skills to students. The Ministry has indicated 
that it shared these findings with schools in 
the Ontario Focused Intervention Partnership 
program. 

The Ministry advised us that it analyzes final 
report-backs from the school boards to assess 
whether board program funding has been allocated 
to the areas of greatest need. In our 2009 audit 
we noted that for almost 70% of the funding over 
the previous five years, either the funding had 
been based on student enrolment or the Ministry 
could not fully explain its allocation method. In the 
2010/11 fiscal year, the Ministry indicated that the 
majority of its program funding was now allocated 
on the basis of need. In addition, the Ministry 
indicated that it had instituted further account- 
ability measures for school boards by specifying the 
funding criteria, reporting requirements, payment 
schedules, and allowable expenditure categories 
in transfer payment contracts to help ensure that 
funds were spent for the purposes intended. 

The Ministry advised that it recently approved 
and released Administration Fee and Lead Board 
Guidelines with criteria for transfer-payment 
expenditures to third-party organizations, such 



as the Council of Ontario Directors of Education 
(CODE) and boards that act as lead boards. 

In the guidelines, clarification is provided with 
respect to the overall rationale for the selection 
of lead boards, and the criteria that need to be 
considered and included in determining whether 
funding to a lead board is required. These include: 

• whether it would be more efficient and effect- 
ive in delivering the services; 

• whether it would provide a more economical 
delivery to ensure value for money; and 

• whether specific expertise resides with the 
lead board. 

CONSISTENCY OF STUDENT 
ASSESSMENTS 

Recommendation 4 

To help ensure that students are being assessed in 
a consistent way, the Ministry of Education should 
monitor the results from different types of assessment, 
especially those from report cards and Education 
Quality and Accountability Office (EQAO) tests, to 
identify any major discrepancies for follow-up. 

Status 

In March 2010, the Ministry completed a study 
that examined the correlation between report 
card marks and EQAO results for grades 3 and 6. 
The study found that report card marks and EQAO 
results appear to be reasonably well aligned and 
that classroom-based assessments are relatively 
consistent over time. It also found that report card 
marks in the two years prior to the EQAO assess- 
ments are consistent predictors of EQAO results. 

The study found that the relationship between 
report card marks and EQAO results was virtually 
the same for grades 3 and 6, and that assessment 
practices of EQAO and teaching to the curriculum 
were relatively well aligned. The study also found 
that report card marks were more closely aligned 
with EQAO results for female than for male 
students. 
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The study also looked into variations between 
report card marks and EQAO results based on 
grade level, curriculum, and student gender. The 
Ministry noted that there will be some variation in 
these assessments, because the nature and purpose 
of the large-scale EQAO assessment are very dif- 
ferent from those of classroom assessment. The 
EQAO assessment reflects a point in time, whereas 
report cards summarize a longer period of time and 
include consideration of greater specific circum- 
stances pertaining to the student. 

The Ministry is reviewing these findings and 
further reviewing its capacity to perform similar 
statistical analysis in the future. 

ONTARIO STATISTICAL NEIGHBOURS 
INFORMATION SYSTEM 

Recommendation 5 

To ensure that all school boards and schools can 
obtain useful and relevant information to develop 



strategies for improving student achievement, the 
Ministry of Education should consider granting them 
direct access to the Ontario Statistical Neighbours 
information system. This would be more cost-effective 
than school boards having to develop and maintain 
their own systems. 

Status 

The Ministry indicated that it has completed the 
development of the online Ontario Statistical Neigh- 
bours system, which gives school boards direct 
access to useful information necessary for improving 
student achievement. At the time of our follow-up, 
50 of the 60 English district school boards and 11 
of the 12 French district school boards had received 
training from the Ministry. The Ministry expected 
the training to be complete in fall 2011. 
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Follow-up on VFM Section 3.08, 2009 Annual Report 



Background 



OntarioBuys is a government initiative launched in 
2004 to help the broader public sector (BPS) save 
money when procuring goods and services. The 
BPS Supply Chain Secretariat, part of the Ministry 
of Finance (Ministry), is responsible for administer- 
ing and managing OntarioBuys. 

Since the 2004/05 fiscal year, OntarioBuys has 
provided funding of about $185 million for two 
areas: about $116 million as of March 31, 2011 
($88 million at the end of the 2008/09 fiscal year), 
for the formation and/or expansion of collaborative 
groups called "shared-service organizations" (SSOs) 
whose members are BPS institutions; and $69 mil- 
lion as March 31, 2011 ($61 million at the end of 
the 2008/09 fiscal year), for 54 projects aimed at 
helping BPS institutions become more efficient and 
effective in their supply-chain and other back-office 
processes. In the 2010/11 fiscal year, the BPS Supply 
Chain Secretariat incurred total direct operating 
expenses of $4.4 million ($4.8 million in 2008/09). 

In our 2009 Annual Report we reported that, 
although the government announced in its 
March 2009 Budget that OntarioBuys had helped 
BPS organizations — mostly from the hospital sec- 
tor — redirect $45 million in savings toward front- 
line services, almost $20 million of this reported 
amount was not redistributed to hospitals to 
provide front-line services. Rather, it was retained 
by the SSO that reported the savings to develop 



information technology for its back-office pro- 
cesses. The balance of the reported savings came 
from a number of projects; however, OntarioBuys 
had not verified these savings nor was it able to 
demonstrate that they had actually been invested in 
front-line services. 

We also found that, although OntarioBuys had 
undertaken significant efforts to promote its col- 
laborative supply-chain initiatives, participation in 
the SSOs, particularly in the education sector, was 
well below the level required for OntarioBuys to 
achieve its goals. Our other significant observations 
included: 

• OntarioBuys approved funding for projects 
on the basis of business cases prepared by 
BPS organizations that included estimated 
costs and potential savings. However, the 
reasonableness of the estimates was often 
not appropriately assessed. For example, the 
largest project approved for funding projected 
savings of $669 million over five years, but we 
found that $294 million of this amount was 
not included in OntarioBuys' funding review 
and that the balance of $375 million was 
determined on an arbitrary basis. Subsequent 
to our audit fieldwork, OntarioBuys revised 
the estimated savings down to $113 million 
over five years. 

• The education SSO, which had received 
$30 million in OntarioBuys funding by 
March 31, 2009, committed to sign up 13 of 
the province's educational institutions and 
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1,000 suppliers by June 2009 to participate 
in a new electronic purchasing system called 
e-Marketplace. As of June 2009, e-Marketplace 
had yet to become operational, and no institu- 
tions had formally signed up to be members. 

• We reviewed a list of project savings that 
OntarioBuys provided us and found some 
purported savings to be questionable. For 
example, our review of savings totalling 
$7.3 million for two projects, which were 
supposed to be completed by December 2006, 
showed that neither project had been com- 
pleted by the time of our audit. Subsequent to 
our review, OntarioBuys revised the total sav- 
ings for the two projects down to $1.1 million. 

• OntarioBuys did not have program-specific 
guidelines for consistent and effective mon- 
itoring of project progress, with requirements 
for conducting site visits, documenting work 
performed, verifying deliverables prior to the 
release of final payments, and closing files for 
completed projects. 

• In the 2004/05 through 2008/09 fiscal years, 
the SSOs and BPS organizations involved in the 
projects spent about $45 million of the funding 
provided to them to hire some 270 consultants 
for a variety of reasons. We reviewed $15 mil- 
lion of consulting contracts from various pro- 
jects and found that over 40% did not comply 
with the competitive procurement require- 
ments of the project funding agreements. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 



Status of Actions Taken on 
Recommendations 



According to the information we received from the 
BPS Supply Chain Secretariat of the Ministry of 



Finance, some progress has been made in address- 
ing all of our recommendations, and substantial 
progress has been made on most of them. 

OntarioBuys informed us that it had embarked 
upon a strategic planning exercise and a review of 
the OntarioBuys program following our 2009 audit. 
It completed the strategic review and finalized the 
strategic plan in September 2010. Following the 
review, OntarioBuys streamlined the program, 
focusing on stabilizing the recently created hos- 
pital and education SSOs to maximize long-term 
benefits, savings, and efficiencies. OntarioBuys also 
developed improved guidelines and provided train- 
ing on its new monitoring processes and the new 
Management Board of Cabinet BPS Procurement 
Directive to OntarioBuys and BPS staff. 

The status of the action taken on each recom- 
mendation at the time of our follow-up was as 
follows. 

APPROVAL OF REQUESTS FOR FUNDING 

Recommendation 1 

To ensure that estimated costs and benefits in busi- 
ness cases are appropriately assessed before being 
approved, OntarioBuys should: 

• obtain the necessary supporting materials from 
applicants to appropriately assess the reason- 
ableness of projected savings and estimated 
costs; and 

• address identified risks and document actions 
taken or to be taken to mitigate these risks. 

Status 

OntarioBuys informed us that it had developed new 
guidelines, processes, and tools for assessing busi- 
ness cases to help it ensure that the projected costs 
and savings are reasonable. All documentation must 
be completed before the business case review can 
be signed off by OntarioBuys senior management. 
Before signing off, senior management must review 
the completed business case guidelines template 
and supporting documentation, which includes 
assessments of the costs, benefits, and risks. 
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Business cases for projects requesting more 
than $1 million in funding are still required to be 
reviewed by the Ministry of Finance's Internal Audit 
Services. If OntarioBuys considers it necessary 
Internal Audit can also be asked to review projects 
with less than $1 million in funding. 

In March 2011, Internal Audit released a report 
that summarized its reviews of funding requests 
in excess of $1 million for the 2010/11 fiscal year. 
The report noted that a common issue identified 
in its review of project proposals was a need for 
greater clarity and detail in the business case and 
supporting documentation in three areas: financial 
information, specifically the defining of baselines 
and targets; documentation to support savings; and 
identification of relevant risks. 

OntarioBuys informed us that it is taking a num- 
ber of actions to address Internal Audit's areas of 
concern. One of these is referencing the documen- 
tation supporting the savings in transfer-payment 
agreements. Before Internal Audit's report, 
OntarioBuys had already developed a standardized 
risk management template to help it identify risks, 
to document risk analyses and risk mitigation, 
and to track actions taken for the risks identified. 
OntarioBuys informed us that, to improve its risk 
management template, it had begun working with 
Internal Audit to develop a more detailed risk 
analysis framework to use in assessing future busi- 
ness cases. 

MONITORING OF FUNDED PROJECTS 
AND EXPENSE CLAIMS 

Oversight of the Status of SSOs and 
Projects and the Achievement of 
Deliverables 

Recommendation 2 

To ensure that the shared-service organizations 
(SSOs) and projects that OntarioBuys funds achieve 
contract deliverables and that funds are used for the 
intended purpose, OntarioBuys should: 



• develop monitoring guidelines to assist its staff 
in consistently conducting appropriate oversight 
of the SSOs and projects funded; and 

• monitor, on a timely basis, the progress of 
funded SSOs and projects against contract deliv- 
erables and take appropriate action when there 
are significant delays. 

Status 

OntarioBuys indicated that in 2010 it had 
developed new Project Monitoring Guidelines 
outlining the tools and processes necessary to 
effectively monitor the required deliverables 
throughout the various phases of a project. The 
guidelines include guidance on the type of informa- 
tion required in project status reports, management 
approval requirements, and the extent of review 
work that staff should do based on the assessment 
of risks, savings, costs, and other benefits. A project 
monitoring flowchart of the review and approval 
process for release of funds was also developed. 
OntarioBuys staff were trained on the new guide- 
lines and processes in July 2010. 

To monitor the progress of projects, OntarioBuys 
has developed an electronic transfer-payment 
tracking system, which was fully implemented by 
September 2010. The information that the system 
tracks includes the progress of funded projects, 
funding flows to transfer-payment recipients, 
transfer-payment forecasts, the names of the 
members of staff/managers who approved a busi- 
ness case or release of funding, and which staff are 
responsible for monitoring which projects. 

Consulting Services 

Recommendation 3 

To ensure that significant consulting-service contracts 
are awarded in an open, fair, and transparent man- 
ner, OntarioBuys should monitor broader-public- 
sector compliance with the required procurement 
policies. 
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Status 

OntarioBuys indicated that, since May 2010, it has 
required BPS organizations to complete procure- 
ment plans for projects identifying all consulting 
procurements to occur over the 12 months follow- 
ing a BPS agreement with OntarioBuys. The BPS 
organization must include in its plan a description 
of the consulting procurement, the estimated 
procurement value, the procurement method to 
be used, the proposed approval authority level, 
and the approximate month and year it expects to 
commence the procurement process. It must also 
indicate whether the procurement could involve 
extensions or future related contracts. 

The BPS organization is also to submit regular 
status reports, called procurement reports, that 
include the following information: procurement 
description; vendor name; final contract price; 
amount paid to date; procurement method applied; 
effective date of the contract; contract deliverables 
achieved; the name of the person approving the 
contract; and any modifications to the contract 
terms. 

OntarioBuys informed us that its staff uses a 
risk-based approach to carefully review procure- 
ment reports, identifying any discrepancies for 
follow-up and applicable corrective actions, and 
comparing the reports with their corresponding 
procurement plans. The reports are also checked 
for compliance with Management Board of Cab- 
inet's BPS Procurement Directive. 

Review and Approval of Expense Claims 

Recommendation 4 

To ensure that only appropriate expenses are reim- 
bursed, OntarioBuys should provide the management 
of shared-service organizations and supply-chain- 
improvement projects with guidelines on the reim- 
bursement of meal, travel, and hospitality expenses, 
with maximum limits that are reasonable when 
compared to those for Ontario government employees. 



Status 

OntarioBuys indicated that, in March 2010, it 
developed a Guideline for Transfer Payment Project 
Expenses for Travel, Meals, and Hospitality requiring 
all transfer-payment recipients receiving funds from 
OntarioBuys to follow the applicable sections of the 
Management Board of Cabinet Travel, Meal and 
Hospitality Expenses Directive. The guideline was 
incorporated into the transfer-payment agreement 
schedules for organizations receiving funding from 
OntarioBuys. OntarioBuys also provided training 
on these expense requirements to transfer-payment 
recipients in April and May 2010. 

Under the authority of the new Broader Public 
Sector Accountability Act, 2010, Management Board 
of Cabinet developed a Broader Public Sector 
Expenses Directive, effective April 1, 2011. This 
directive replaced the OntarioBuys Guideline for 
Transfer Payment Project Expenses for Travel, Meals, 
and Hospitality, and OntarioBuys requires BPS 
organizations to make reimbursements in accord- 
ance with the new directive. 

The directive requires the designated BPS 
organizations to establish rules for expenses that 
are reimbursed from public funding and sets out 
what organizations need to consider when they are 
establishing their rules for expenses. BPS organiza- 
tions were trained on the new directive in February 
and March 2011. 

PERFORMANCE MEASUREMENT 

BPS Participation in SSOs and Projects 

Recommendation 5 

To assist Ontario educational institutions to more 
effectively generate savings from improved supply- 
chain-management practices, OntarioBuys should 
more formally assess the impact of the various col- 
laborative purchasing initiatives already in place in 
the education sector on the effectiveness of the educa- 
tion shared-service organization (SSO) and assess 
whether any changes are necessary to the education 
SSO's business model. 
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Status 

OntarioBuys indicated that it had determined 
through a formal review of the various collab- 
orative purchasing initiatives and the level of 
participation of educational institutions that a 
shared-service organization (SSO) for the educa- 
tion sector continued to be a viable option. We were 
also informed that, after our 2009 audit, the educa- 
tion SSO eliminated the development and imple- 
mentation of an electronic purchasing system and 
focused instead on strategic sourcing and collabora- 
tive purchasing initiatives. The SSO submitted a 
new business case reflecting these changes, and 
OntarioBuys approved it. A new transfer-payment 
agreement was signed at the end of June 2011. 

Reported Savings 

Recommendation 6 

To ensure that reported performance results are cred- 
ible, OntarioBuys should: 

• provide guidelines to shared-service organiza- 
tions (SSOs) and broader-public-sector institu- 
tions on how savings are to be defined and how 
baselines are to be established and applied for 
the calculation of savings; and 

• objectively assess and verify SSOs' and projects' 
reported savings to ensure that they are valid 
before publicly disclosing them as results 
achieved. 

Status 

OntarioBuys informed us that it had distributed 
new guidelines to its funded transfer-payment 
recipients and shared-service organizations in 
July 2010 to help them calculate and validate 
forecasted and actual savings for funded projects. 
OntarioBuys also indicated that it conducts periodic 
spot checks to assess whether projects' actual sav- 
ings are defined in a manner that is consistent with 
the guidelines. 



OntarioBuys has not published project savings 
since our 2009 audit. OntarioBuys indicated that 
it would ensure that reported savings are verified 
in accordance with the savings guidelines prior to 
public disclosure and that appropriate documenta- 
tion is maintained. 

Other Performance Measures 

Recommendation 7 

To properly measure and report on performance 
results, OntarioBuys should: 

• use actual purchase information from funded 
shared-service organizations (SSOs) to 
determine whether it has achieved the target 
percentage of having certain supplies purchased 
through them; and 

• develop performance measures and collect the 
information necessary to assess and report on 
the redirecting of savings generated by funded 
SSOs and projects to front-line services. 

Status 

OntarioBuys informed us that after our 2009 audit 
it determined that it would be unable to obtain 
annually comparable information on total spend- 
ing in each sector to assess whether the target 
percentage of savings expected from having certain 
supplies purchased through the SSOs was actually 
achieved. 

At the time of our follow-up, OntarioBuys was 
exploring the types of data necessary to determine 
more appropriate measures of results and had 
developed a new set of draft performance measures 
that it expected to finalize by September 2011. The 
timing and development of the related baselines 
and targets is to depend on the performance meas- 
ures that OntarioBuys ultimately selects. 
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Ontario Disability 
Support Program 



Follow-up on VFM Section 3.09, 2009 Annual Report 



Background 



The Ministry of Community and Social Services 
(Ministry) administers the Ontario Disability Sup- 
port Program Act (Act), which provides income and 
employment support to more than 270,000 individ- 
uals with eligible disabilities as defined by the Act. 
Total annual Ontario Disability Support Program 
(ODSP) benefit payments made in the 2010/11 fis- 
cal year amounted to over $3.5 billion ($3 billion in 
2008/09 — which was a 42% increase since the time 
of our last audit in 2004) . 

ODSP income support is intended to assist with 
basic living expenses such as food, shelter, clothing, 
and personal-needs items. Although employment- 
support programs are available to ODSP recipients, 
participation in them is not required. As a result, 
relatively few ODSP recipients join such programs. 

In our 2009 Annual Report, we found that 
although the Ministry had implemented a num- 
ber of the recommendations contained in our 
2004 Annual Report, there had been only limited 
improvements in determining an applicant's finan- 
cial eligibility and the amount of assistance to be 
paid. 

The Ministry had established a two-stage pro- 
cess to ensure that only qualified applicants receive 
income support. The first stage relied solely on 



applicants volunteering financial information. To 
compensate for the risks associated with this, the 
second stage was third -party verification of certain 
information provided by the applicant. However, 
this verification was largely ignored in practice. As 
a result, the Ministry was not adequately ensur- 
ing that only eligible individuals were receiving 
payments in the correct amounts. Some of the 
issues identified in our 2009 Annual Report were as 
follows: 

• Although the Ministry had significantly 
reduced the average wait time for a medical- 
disability determination decision, 60% of 
recipients sampled still received late pay- 
ments. On average, they experienced a 58-day 
delay after they had been determined to be 
medically qualified for payments, which was 
almost three times longer than the outside 
limit of 21 days established by the Ministry. 
These delays in receiving approved bene- 
fits offset to a significant degree the good 
progress made since our 2004 audit in exped- 
iting the initial medical determination. 

• Oversight procedures were lacking with 
regard to monitoring and assessing the 
fairness and consistency of decisions made 
by individual adjudicators at the Ministry's 
Disability Adjudication Unit (DAU). Con- 
sequently, the rates at which adjudicators 
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determined that applicants were eligible 
generally varied from 11% to 49%. 

• In the 2008/09 fiscal year, 55% of applicants' 
appeals to the Social Benefits Tribunal led 

to the Tribunal overturning the Ministry's 
initial decision to not approve an applicant for 
benefits. 

o Since 2002, the Ministry had not performed 
any of the periodic medical reassessments — 
required by legislation — to ensure continuing 
eligibility for disability support payments. 

• The Ministry relied on one individual to do all 
the assessment and reassessment work for any 
given file, yet the individual's work was nei- 
ther supervised nor reviewed to ensure that 
the decisions made complied with ministry 
and legislative requirements. 

• The total amount of overpayments for both 
active and inactive accounts had increased 
substantially, from $483 million in 2004 to 
$663 million as of March 31, 2009. In many 
cases, overpayments resulted from what 
would appear to be recipients fraudulently 
misrepresenting their circumstances. These 
overpayments might often have been avoided 
if the Ministry had more effectively reassessed 
the eligibility and the amounts to be paid to 
those individuals identified by its own systems 
as high-risk or followed up on tips received 
from the public. 

• The Ministry's computerized Service Delivery 
Model Technology (SDMT) information 
system still lacked key internal controls, and 
regional and local offices were not receiv- 
ing, in an easily understandable format, the 
information they needed to effectively oversee 
program expenditures. 

STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this audit in May 2010. In Novem- 
ber 2010, the Committee tabled a report in the 



Legislature resulting from this hearing. The report 
contained nine recommendations and requested 
the Ministry to report back to the Committee with 
respect to the following: 

• whether the Ministry had begun meeting its 
own target that ODSP clients receive their 
cheques within 21 days of being approved 
for benefits and, if not, how long on average 
clients were waiting to receive their cheques 
(the Committee also asked the Ministry to 
consider posting this information on its ODSP 
website) ; 

• what progress had been made in the area of 
oversight and review of adjudicator decisions 
and an assessment of the effectiveness of 
the Ministry's new process for oversight and 
review of adjudicator decisions, including an 
estimate of the percentage of files that get 
reviewed; 

• the outcome of the Ministry's consideration 
of possible strategies for addressing the 
Social Benefits Tribunal's high overturn rate, 
specifically: 

• the current overturn rate; 

• whether the Ministry had established a 
target for the overturn rate; 

• whether it had introduced measures to 
ensure that Tribunal members and Ministry 
staff are using the same criteria to deter- 
mine disability and make income support 
decisions, what these criteria are, and how 
their use is enforced; and 

• whether it had examined the eligibility 
adjudication process for the Canada Pen- 
sion Plan Disability benefits (and if not, the 
Committee asked it to provide a rationale) ; 

• the outcome of the Ministry's review of 
business processes for processing fraud tips, 
including what measures it would be introdu- 
cing to better identify and deal with suspected 
fraud cases on a more timely basis and current 
metrics on phone tips, police referrals, convic- 
tions, and data on trends; 
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• whether the Ministry was addressing its back- 
log of required medical eligibility reassess- 
ments, specifically: 

• whether it would be increasing its rate of 
medical reviews from the current rate of 
100 reviews per month; 

• the most recent review results; and 

• whether it would be requesting additional 
staff to catch up on the backlog; 

• an update of the Ministry's discussions with 
the Ministry of Health and Long-term Care 
on the design and implementation of the new 
nutritional supplement program that was to 
replace the special dietary allowance; 

• with respect to overpayments and client debt: 

• how much of the $663 million in overpay- 
ments the Ministry realistically expected 
to collect and how much should be recom- 
mended to be written off; 

• the outcome of its plan to develop a more 
robust writeoff strategy for client debt; 

• whether it would be holding discussions 
with the Ministry of Finance to ensure that 
any writeoff strategy that it develops will 
comply with the rules set by that Ministry, 
and if so, when; and 

• its assessment of the root causes for over- 
payments, including the decisions it had 
taken on how it will address this issue; 

• the main features of its new front-line service 
delivery model, how this model promotes 
efficient service delivery, whether it had 
made an impact on the number of staff that 
are required to deliver front-line service, and 
whether the number of sick days taken by 
caseworkers still averages 20 days per year; 
and 

• what progress the Ministry had made in 
developing a business case for the replace- 
ment of its Service Delivery Model Technol- 
ogy, specifically: 

• how consultations with caseworkers would 
be incorporated into the process to ensure 
that the new system meets user needs; 



• whether the new system was on track to 
be implemented by the end of the 2012/13 
fiscal year; 

• what progress, if any, had been made in 
determining whether an off-the-shelf 
system would be suitable and, if a suit- 
able system were found, what changes to 
the Ministry's work processes would be 
required to be able to use such a system; 
and 

• what progress, if any, had been made in 
sequestering management access from 
caseworker access in the system that the 
Ministry is currently using. 

The Ministry responded to the Committee in 
March 2011. A number of the issues raised by the 
Committee were similar to our observations. Where 
the Committee's recommendations are similar to 
ours, this follow-up includes the recent actions 
reported by the Ministry to address the concerns 
raised by both the Committee and our 2009 audit. 



Status of Actions Taken on 
Recommendations 



According to information received from the Min- 
istry between May and September 2011, progress 
has been made in addressing most of the recom- 
mendations in our 2009 Annual Report, with 
substantial progress having been made on a few 
of them. However, more effort and time will be 
needed before the Ministry is able to fully imple- 
ment all of our recommendations. For example, fur- 
ther progress will depend on the implementation of 
several initiatives the Ministry currently has under 
way, such as recruiting and training 120 new front- 
line staff to improve service and to help cope with 
increased pressures and rising caseloads owing to 
the recent economic downturn; reorganizing and 
training staff with new tools to strengthen program 
management and oversight; and implementing a 
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new information technology system. In addition, 
in November 2010, the government announced the 
launch of a major review of the social assistance 
programs in Ontario. 

The status of actions taken on each of our rec- 
ommendations at the time of our follow-up was as 
follows. 

INITIAL FINANCIAL ELIGIBILITY 
ASSESSMENT 

Recommendation 1 

To ensure that an individual's initial financial eligibil- 
ity for Ontario Disability Support Program benefits is 
adequately verified, the Ministry of Community and 
Social Services should: 

• comply in all cases with its own requirements to 
verify an applicant's declared income and assets 
with the third parties who have information- 
sharing agreements with the Ministry; and 

• conduct supervisory reviews, at least on a 
sample basis, of the decisions made and files 
maintained by intake caseworkers to ensure 
that staff are adhering to Ministry requirements 
with respect to financial eligibility verification. 

Status 

The Ministry indicated that it has developed a new 
standardized form for third-party verifications in its 
Service Delivery Model Technology (SDMT) system 
to document and maintain a record of the results of 
each third-party check conducted. This new form is 
expected to provide a consistent approach to docu- 
menting third-party checks across the province and 
to make it easier for staff to find information related 
to third-party checks conducted. The Ministry also 
advised us that it has provided optional training to 
its staff on the Equifax credit reports — a third-party 
verification procedure — to assist them with reading 
and understanding the reports. 

The Ministry also informed us that in April 2010, 
it implemented ODSP file reviews using a standard- 
ized checklist, and managers have been conducting 
these file reviews since that time. The results of the 



first round of these reviews were evaluated, with the 
outcome that approximately 60% of the files were 
found to have no issues with the decisions made 
or how the files were maintained. However, for the 
remaining 40% of files, issues were noted, such 
as third-party checks not being documented and 
required documents not being on file, which were 
similar to the findings in our 2009 Annual Report. 
The Ministry has since reinforced with its staff the 
requirements for the areas where issues were noted. 

INITIAL DISABILITY DETERMINATION 

Recommendation 2 

To ensure that all Ontario Disability Support Pro- 
gram applicants are adjudicated fairly and consist- 
ently, the Ministry of Community and Social Services 
should: 

• periodically review a random sample of each 
adjudicator's files to assess whether the deci- 
sions are generally supported and fair; and 

• monitor the percentage of applicants found to 
have an eligible disability by each adjudicator 
and, if there are significant variances, investi- 
gate the reasons for them and take corrective 
action where necessary. 

Status 

The Ministry informed us that in March 2010 a 
formal adjudication file review process was estab- 
lished whereby now the Manager of Adjudications 
and Medical Policy reviews a sample of approxi- 
mately 40 adjudicator files each week to determine 
the appropriateness of the decisions made and to 
identify any training needs. A file feedback form is 
to be completed for each review conducted and is 
provided to the applicable adjudicator at the end 
of the review. In addition, for any file reviewed 
where it is recommended that the original deci- 
sion be overturned, the file is further reviewed by 
a panel of three individuals who then make a final 
determination. The Ministry indicated to us that 
the average overturn rate as a result of the reviews 
completed to date was approximately 7%. 
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The Ministry now also regularly monitors per- 
formance reports for each adjudicator, including 
statistics on adjudications, and additional files may 
be selected for review on the basis of the results 
reported. When any systemic or ongoing issues are 
identified from the file review process, corrective 
action, such as group or targeted training, is taken. 

SOCIAL BENEFITS TRIBUNAL APPEALS 

Recommendation 3 

To reduce the need for, and cost of, appeals and the 
relatively high rate at which the Social Benefits Tri- 
bunal overturns Ontario Disability Support Program 
eligibility decisions, the Ministry of Community and 
Social Services should consult and work with the 
Tribunal to narrow the differences in approach to, 
and criteria used in, assessing individuals with a 
disability. In addition, to ensure that its rationale for 
denying a claim is clearly communicated to the Tribu- 
nal, the Ministry should ensure that it is represented 
by a case-presenting officer at every hearing. 

Status 

The Ministry informed us that it has entered into 
a new memorandum of understanding with the 
Social Benefits Tribunal to clarify the accountability 
relationships between the two parties, and it is 
conducting in-depth reviews and analysis twice a 
year on the Tribunal's disability-related decisions. 
Although the Tribunal's rate of overturning min- 
istry adjudication decisions is still similar to that 
at the time of our audit, the Ministry has shared its 
policies and other information relating to its adjudi- 
cation process with the Tribunal. However, the 
Ministry indicated that the Tribunal is an independ- 
ent body that operates at arm's length from the 
Ministry and sets its own policy and operational 
direction. 

In addition, the Ministry informed us that it is 
currently unable to ensure that it is represented by 
a case-presenting officer at every hearing due to 
resource limitations. 



ELIGIBILITY REASSESSMENTS/ 
CONSOLIDATED VERIFICATION PROCESS 

Financial Eligibility Reassessments 

Recommendation 4 

To ensure that recipients continue to be financially eli- 
gible for Ontario Disability Support Program benefits 
and to avoid overpayments, the Ministry of Commun- 
ity and Social Services should: 

• ensure that recipients identified as high-risk are 
prioritized for review; 

• comply in all cases with its own requirement to 
verify an applicant's declared income and assets 
with the third parties with whom the Ministry 
has information-sharing agreements; and 

• be more proactive in following up on those tips 
that come from what appear to be bona fide 
sources. 

Status 

As noted in its response to our 2009 recommenda- 
tion, as an interim measure the Ministry continues 
to review a limited sample of cases for financial 
eligibility based on various periodically assessed 
risks — for example, Canada Revenue Agency 
data matches. However, it informed us that it has 
developed a new risk-based eligibility reassessment 
process in conjunction with Equifax Canada called 
the Eligibility Verification Model. This new process 
is expected to assist in the identification and priori- 
tization of high-risk cases for eligibility reviews by 
linking ODSP data with Equifax consumer credit 
databases. Testing of the new process began in 
October 2010; however, it had not yet been imple- 
mented at the time of our follow-up. 

As noted earlier, the Ministry indicated that it 
has developed a new standardized form for third- 
party verifications in its SDMT system to document 
and maintain a record of the results of each third- 
party check conducted. This new form is expected 
to provide a consistent approach to documenting 
third-party checks across the province and to make 
it easier for staff to find information related to 
third-party checks conducted. The Ministry also 
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advised us that it has provided optional training to 
its staff on the Equifax credit reports — a third-party 
verification procedure — to assist them with read- 
ing and understanding the reports. However, the 
results of recent file reviews undertaken by ODSP 
managers indicated that not documenting third- 
party checks remains an issue. 

The Ministry informed us that in order to be 
more proactive in following up on bona fide tips, 
it has instituted a 15-day standard for ODSP staff 
to complete a preliminary assessment of all tips 
received. However, it has not yet verified that the 
new standard is being met. The Ministry also under- 
took a review to identify best practices in fraud pre- 
vention and detection from other jurisdictions (such 
as other provinces and the United States) and from 
its service managers across Ontario. Recommenda- 
tions were expected in fall 2011. 

Management of Outstanding Tasks 

Recommendation 5 

To ensure that Ontario Disability Support Program 
benefits are paid only to eligible individuals and in 
the correct amount, the Ministry of Community and 
Social Services should monitor case-management 
activities to ensure that tasks entered into its Service 
Delivery Model Technology information system are 
followed up on promptly and that appropriate actions 
are taken to avoid overpayments. 

Status 

The Ministry advised us that it undertook a cleanup 
exercise for outstanding tasks in its computer sys- 
tem whereby it removed 40% of all open tasks from 
the system after determining that they were redun- 
dant. With regard to new tasks being created, we 
were also advised that the Ministry simplified the 
programming so that tasks considered unnecessary 
are no longer generated automatically. In addition, 
the Ministry now prepares monthly reports that it 
sends to its managers to assist in identifying over- 
due tasks on which action must be taken promptly. 



Medical Eligibility Reassessments 

Recommendation 6 

To comply with the Ontario Disability Support 
Program Act and to ensure that only eligible ODSP 
recipients continue to receive benefits, the Ministry of 
Community and Social Services should conduct the 
required medical reassessments within the legislated 
timeframe. 

Status 

The Ministry advised us that in May 2009, it 
began conducting medical reassessments. Initially, 
approximately 100 cases per month were selected 
for reassessment from among those recipients 
with a reassessment date within the last two years. 
Review packages were sent to the selected recipi- 
ents to be completed by an approved health-care 
practitioner and returned to the Ministry within 90 
days. 

After conducting the reassessments for a year, 
the Ministry undertook an evaluation of the process 
to assess its effectiveness and identify areas for 
improvement. The evaluation included an assess- 
ment of the results of the reassessments conducted 
and a survey of staff on their experience to date. 
The results of the reassessments indicated that out 
of 1,553 reviews conducted, approximately 1,077 
packages were returned; of those, approximately 
76% were confirmed to be still eligible for benefits, 
and 24% were no longer considered eligible. Bene- 
fits for individuals who did not return their review 
packages were to be suspended until the completed 
review package was received. 

The Ministry also informed us that at the 
time of our follow-up, medical reassessments for 
approximately 28,400 recipients were overdue, 
which represents 45% of all recipients requiring 
a medical reassessment. We were informed that 
as of July 2011, medical reassessments had been 
temporarily suspended due to an increase in new 
applications received and the need for staff to pro- 
cess those applications. 
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Income-support Payments to Individuals 

Recommendation 7 

To ensure that eligible applicants receive the correct 
financial entitlements within a reasonable time 
frame, the Ministry of Community and Social Services 
should ensure that: 

• Ontario Disability Support Program payments 
start within the prescribed 21 calendar days 
of the determination that the person has an 
eligible disability; 

• all of the information necessary to determine 
the correct amount of benefits is on file and 
correctly considered before payments are made; 
and 

• suspicious or unusual circumstances, including 
those relating to the special dietary allow- 
ance, are appropriately flagged for additional 
follow-up. 

Status 

At the time of our follow-up, the Ministry had not 
yet ensured that payments start within the pre- 
scribed time period and that all information neces- 
sary to determine the correct amount of benefits is 
on file. However, it has begun hiring an additional 
120 new front-line staff and has provided extensive 
training for staff, which over time should help in 
these regards. 

The Ministry has also implemented a new 
standardized file review process using standardized 
checklists and tracking tools, to help determine 
compliance with program requirements. 

With regard to the special dietary allowance, the 
Ministry undertook a forensic audit to determine 
the extent of possible misuse of the allowance, 
which corroborated many of the findings in our 
2009 Annual Report. In March 2010, the govern- 
ment announced plans to eliminate the special 
dietary allowance and create a new medically 
based nutritional supplement program for social 
assistance recipients with severe medical needs that 
would be administered by the Ministry of Health 
and Long-Term Care. However, in November 2010, 
the government announced that the special dietary 



allowance would continue, but would be revised to 
comply with an earlier order of the Human Rights 
Tribunal of Ontario and to address the recommen- 
dations of an expert committee. 

The changes to the administration of the special 
dietary allowance, which took effect in April 2011, 
included the following: 

• removing conditions that the expert com- 
mittee found to not require a special dietary 
allowance; 

• revising the application form to require 
recipients to consent to the release of relevant 
medical information by their physician to sup- 
port their application; 

• requiring ODSP recipients to reapply for the 
special dietary allowance, which has resulted 
in a drop of about 23,000 cases receiving the 
allowance, or a funding impact of about $2 mil- 
lion per month; 

• filing complaints with the College of Phys- 
icians and Surgeons where deemed appropri- 
ate; and 

• confirming that ODSP staff have the legisla- 
tive authority to determine eligibility for the 
allowance, including the authority to request 
additional information or deny an application 
in cases where the information provided is 
believed to be false or incorrect. 

In addition, the Ministry began to use its infor- 
mation technology system to help identify question- 
able trends in a timely manner so that appropriate 
action could be taken to limit potential abuse. 

These changes will improve the administration 
of the special dietary allowance as long as the Min- 
istry ensures that all staff are complying with them. 

OVERPAYMENTS 

Recommendation 8 

To better utilize its limited resources and help maxi- 
mize the recovery of Ontario Disability Support Pro- 
gram overpayments, the Ministry of Community and 
Social Services should: 
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• devote more efforts to minimize overpayments 
in the first place, given the limitations in 
recovering significant overpayments from active 
and inactive recipients; 

• ensure that overpayments from inactive 
accounts are transferred to, and followed up on 
by, the Ministry's Overpayment Recovery Unit 
on a timelier basis, with emphasis on accounts 
that are considered to have the most potential 
for repayment; and 

• assess the validity and collectibility of outstand- 
ing overpayments designated as temporarily 
uncollectible and, where warranted, recommend 
that they be written off so that attention can 

be focused on those accounts where collection 
efforts are likelier to yield results. 

Status 

The Ministry advised us that in an attempt to mini- 
mize overpayments, it has enhanced its information 
technology system to include a new Benefit Unit 
Entitlement Report. The report provides a detailed 
history of a recipient's entitlements, program eligi- 
bility details, and overpayments, which will make it 
easier for staff to understand why the overpayment 
occurred and to verify the amount. The Ministry 
expects that the use of this report will assist staff 
in making more timely eligibility assessments 
and detecting issues earlier, thereby helping to 
minimize overpayments. Full-day training was also 
provided to staff on overpayment processes and 
referrals to the Overpayment Recovery Unit (ORU) . 

The Ministry informed us that it has made some 
improvements to help ensure that overpayments 
from inactive recipients are transferred to, and 
followed up on, by the ORU on a timelier basis. 
For example, it has begun to electronically transfer 
overpayments to the ORU, allowing for a more 
timely transfer of data between the two offices and 
reducing the time spent on manual data entry. In 
addition, the ORU now accepts payment by pre- 
authorized debit, to make it easier and faster for 
payments to be made and reduce the frequency of 
paper payments and dishonoured payments. The 



ORU has also increased its own efficiency to free 
up staff to focus more on collection efforts — by, for 
example, combining notification letters to reduce 
the referral time to the Canada Revenue Agency, 
and enhancing its database to eliminate some 
labour-intensive processes. 

With regard to assessing the validity and collect- 
ibility of overpayments and writing off those that 
are warranted, the Ministry established the Social 
Assistance Overpayment Recovery Working Group 
in February 2010 to thoroughly review its overpay- 
ment policies and recovery practices, to research 
industry standards, and to develop strategies for 
improving the Ministry's current collection efforts. 
A report was issued in December 2010 and an 
implementation plan was subsequently developed 
that resulted in, among other things, the writeoff 
of approximately $118 million in uncollectible 
overpayments. The writeoff of these uncollectible 
accounts should allow the Ministry to better focus 
its collection efforts on accounts that have a better 
chance of being collected. 

Then in March 2011, the Ministry undertook a 
review of the collectibility of the remaining over- 
payments to determine if further accounts could 
be written off, but at the time of our follow-up no 
additional accounts had been written off. 

CASE MANAGEMENT 

Workload 

Recommendation 9 

To ensure that Ontario Disability Support Program 
caseworkers can effectively carry out their responsibil 
ities, the Ministry of Community and Social Services 
should: 

• assess caseworkers' responsibilities and work 
processes to establish reasonable caseload 
benchmarks in each of the 44 local offices; and 

• strengthen efforts to monitor sick leave and set 
targets for reducing absenteeism to more reason 
able levels. 
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Status 

At the time of our follow-up, the Ministry had not 
yet established reasonable caseload benchmarks for 
each of its local offices. However, it advised us that 
in order to effectively manage growing caseloads, 
to enhance program integrity, and to improve cus- 
tomer service, a new service delivery and staffing 
model was implemented in January 2011. Before 
the new model was implemented, new province- 
wide business processes were released and 
extensive training was provided. The new model 
included the reorganization of the core ODSP pos- 
itions and the addition of 120 new front-line staff, 
which should help the Ministry to equalize its case- 
loads across its local offices. The Ministry was in the 
process of recruiting across the province to fill these 
new positions. The Ministry has also developed a 



new Operational Indicators Report, which provides 
management with information on caseloads and 
assists in decision-making. 

With regard to strengthening efforts to monitor 
sick leave and set targets for reducing absenteeism, 
the Ministry informed us that it has adopted a case 
management approach to managing staff attend- 
ance, which involves meeting with staff when they 
have incurred six sick days and using monthly 
reports to monitor sick days taken and to identify 
sick-leave issues. The Ministry provided us with 
updated statistics on sick days for the three regions 
whose sick-leave averages for 2008, as noted in our 
2009 audit, were more than 20 days; the average 
for those three regions for 2009 had been reduced 
to 15 days per year. 
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Follow-up on VFM Section 3.10, 2009 Annual Report 



Background 



The Ontario Research Fund (Fund) was created in 
2004 to "support scientific excellence by supporting 
research that can be developed into innovative 
goods and services that will boost Ontario's econ- 
omy." The Ministry of Research and Innovation 
(Ministry), itself created in 2005, is responsible for 
the Fund, which focuses on activities that support 
Ontario's knowledge economy and create high- 
value jobs. 

The Fund makes grants to cover the direct and 
indirect operational costs of research through its 
Research Excellence Program. It supports the cap- 
ital costs of research through its Large Infrastruc- 
ture Program and Small Infrastructure Program. 

Total spending on these programs in the seven 
years between the Fund's inception in 2004 and 
the end of the 2010/11 fiscal year was $569 mil- 
lion ($303 million in the five years from 2004/05 
through 2008/09), with total announced program 
commitments from 2004/05 through 2010/11 of 
$1,077 billion ($623 million from 2004/05 through 
2008/09). The Ministry has approximately 15 staff 
involved in delivering these programs. 

In our 2003 audit of the Science and Technol- 
ogy Division of the former Ministry of Enterprise, 
Opportunity, and Innovation, we reported 
significant concerns about the lack of effective 
governance and accountability mechanisms. The 



consolidation of operating and capital research 
funding into one program managed and admin- 
istered by the Ministry helped address these 
concerns. However, in our 2009 Annual Report, we 
noted that there were still a number of areas that 
required improvement. 

Some of our most significant observations were 
as follows: 

• The Fund's overall mandate emphasizes 
the support of research that will provide 
economic and social benefits for the people 
of Ontario through the commercialization of 
such research. However, most of the $623 mil- 
lion committed to projects at the time was 

for basic theoretical research that was not 
focused on commercial potential. 

• The Ministry measured the performance of its 
projects against three targets: the dollar value 
of investments made by the private sector, the 
number of individuals with enhanced skills 
involved in its projects, and the number of 
active licences for intellectual property rights 
resulting from Ministry-funded projects. How- 
ever, it did not measure the Fund's contribu- 
tion to the overall goal of creating high-paying 
jobs and commercializing research. 

• The Ministry based its Large Infrastructure 
Program funding decisions on the decisions of 
the Canada Foundation for Innovation (CFI) . 
As a result, we noted that the Fund granted 
$41.5 million to projects that did not directly 
support Ontario's strategic priorities. 
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• The Ministry relied on the CFI to monitor 
Research Infrastructure Program grants and 
did not sufficiently assess or review the CFTs 
work to ensure that funding commitments 
worth more than $300 million at that time 
were being spent for the approved purpose. 

• Ontario's colleges tend to focus on applied 
programs and research, and on helping small- 
and medium-sized businesses develop tech- 
nologies and processes for the marketplace. 
However, the Fund had awarded no grants 
directly to colleges. It was our view that the 
Ministry should assess the potential benefits 
of applied-research projects that address both 
the unique needs of Ontario's colleges and 
offer enhanced commercialization potential. 

• The Ministry received reports from grant 
recipients as part of the monitoring process 
for the Research Excellence Program. How- 
ever, we found that the Ministry performed no 
formal monitoring of these reports to ensure 
that program funds were being spent for the 
intended purpose. 



Status of Actions Taken on 
Recommendations 



According to information received from the 
Ministry of Research and Innovation (Ministry), 
some progress was made on implementing all of 
the recommendations in our 2009 Annual Report, 
with substantial progress being made on several, 
including: 

• implementation of a new process called the 
Ontario First approach to ensure that research 
infrastructure projects co-funded with the fed- 
eral Canada Foundation for Innovation (CFI) 
provide strategic benefits to Ontario; 

• launch of the Fund's College-Industry Innova- 
tion Fund to provide co-funding with the CFI 



to meet the research infrastructure needs of 
Ontario colleges; and 
• development of a computer system called 
eRIMS to improve the accountability and 
transparency of the grant application, adjudi- 
cation, and contract-management processes. 
For some recommendations, further progress 
will depend on data collection and reporting of new 
proposed performance measures in late 2011 and 
early 2012, and the development of an information- 
sharing agreement with the CFI that sets out the 
responsibilities of each party in sharing monitoring, 
audit, and site-visit reports. The current status of 
action taken on each of our recommendations is as 
follows. 

PROGRAM OBJECTIVES, BENEFITS, AND 
REPORTING EFFECTIVENESS 

Program Objectives 

Recommendation 1 

To ensure that the Ontario Research Fund (Fund) pro- 
gram supports the Ministry of Research and Innova- 
tion's (Ministry's) overall strategy of job creation and 
is consistent with the Fund's commercialization object- 
ive, the Ministry should place more emphasis on fund- 
ing projects that have viable commercial potential. 

Status 

The Ministry informed us that it continues to 
emphasize commercial potential as one of the key 
assessment criteria for research proposals, although 
it has implemented no new project-application poli- 
cies and procedures. 

Proposals under the Research Excellence 
Program are formally evaluated by peer-review 
panels that include at least two commercialization 
experts who help assess each application's market 
potential, while technology-development proposals 
for the Large Infrastructure Program must provide 
commercialization plans. The importance of poten- 
tial commercialization has also been reiterated in 
program guidelines and project contracts. 
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The Ministry indicated that it does not formally 
track the percentage of funds granted under the 
Research Excellence or Research Infrastructure 
programs that have commercial value. However, 
it continues to collect and report preliminary 
data on patents granted, new and active licences 
established, and spinoff companies created. The 
Ministry also collects anecdotal evidence of com- 
mercialization in the form of success stories shared 
by researchers. 

In 2011, the Ministry began to evaluate a revised 
annual progress report for projects under the 
Research Excellence Program intended to capture 
more information on commercialization achieve- 
ments, including the number of spinoff firms 
created and new employees hired. As more data 
becomes available, the Ministry plans to perform 
more in-depth analysis of research projects to 
compare actual commercialization activity with 
intended goals. 

Benefits of Research Projects 

Recommendation 2 

To better promote the commercialization of research 
done at Ontario's publicly funded research institu- 
tions and ensure that the social and economic benefits 
of the research are retained in Ontario, the Ministry 
of Research and Innovation should continue to review 
best practices for intellectual property management in 
other jurisdictions and, on the basis of the best practi- 
ces identified, implement consistent guidelines for the 
management of intellectual property across Ontario's 
publicly funded research institutions. 

Status 

The Ministry informed us that, in December 2010, 
it consulted with Industry Canada to share infor- 
mation on effective practices in the field, and it 
researched intellectual-property ownership models 
at Ontario universities. As part of the study, the 
Ministry also examined other jurisdictions in Can- 
ada and the United States to identify best practices 
in this field. 



The Ministry concluded from its research that 
approaches vary widely among universities, with 
no consensus as to what works best. The study also 
noted that there is no clear link between intellectual- 
property ownership policy and the rate of commer- 
cialization of research at universities. The Ministry 
informed us that it concluded from its research that 
no single approach is ideal for all situations, so it 
has implemented no standard guidelines in this 
area. However, the Ministry continues to talk to its 
federal and provincial counterparts, and plans to 
encourage the development of intellectual-property 
models and approaches that will maximize the 
benefits to Ontario. 

Measuring and Reporting on Program 
Effectiveness 

Recommendation 3 

To improve its accountability to the public and its abil- 
ity to measure the results being obtained for the grants 
provided by the Ontario Research Fund (Fund), the 
Ministry of Research and Innovation should: 

• develop program-specific measures, targets, and 
benchmarks to assess the Fund's contributions 
to its overall goals of supporting job creation 
and the commercialization of research; and 
m periodically report to the Legislature and the 
public on the achievement of these measures. 

Status 

In 2010, the Ministry engaged a consultant to assess 
the performance of many of its programs and their 
contribution to job creation in Ontario. The report 
found that more than 7,000 jobs created in Ontario 
were attributable directly or indirectly to the Fund 
since its inception in 2004, although the report 
noted that there were gaps in the Ministry's data. 
The report further indicated that bigger programs 
like the Fund contribute proportionately more to 
job creation than smaller ones. 

In 2011, the Ministry contracted an independent 
research firm to collect and analyze the informa- 
tion needed to assess the long-term outcomes of 
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Ministry-funded programs, including the Fund. As 
part of the study, the firm sent an on-line question- 
naire to 1,274 researchers and companies, and 
followed up with 129 interviews with senior rep- 
resentatives of organizations that received direct 
or indirect funding through ministry programs. 
The study noted some key findings attributable to 
ministry funding, including: 

• significant research discoveries or technology 
developments, such as a new process, prod- 
uct, or service; 

• the number of jobs created, with the propor- 
tion that were high-paying and low-paying, 
and the proportion of high-skilled versus low- 
skilled; and 

• the number of spinoff companies created. 

The Ministry anticipates that the proposed per- 
formance measures derived from the study will be 
implemented in fall/winter 2011/12. 

PROJECT SELECTION 

Research Excellence Program 

Recommendation 4 

To ensure that the Research Excellence Program 
follows a selection process that is not only fair and 
transparent but promotes the program's goals, the 
Ministry of Research and Innovation should ensure 
that all approved proposals meet program-eligibility 
requirements. 

Status 

The Ministry informed us that the Fund will no 
longer support projects that fail to meet eligibility 
requirements. In 2009, for example, the Ministry 
excluded two high-performance computing projects 
because they did not meet the Research Excellence 
Program's eligibility requirements. Although the 
projects were important to Ontario researchers, 
the Research Excellence Program's Advisory Board 
recommended that the Program was not the best 
mechanism for funding such projects and sug- 
gested, instead, a separate process to fund propos- 
als that do not strictly meet the Program's eligibility 



requirements. The Minister upheld the Board's 
recommendation, and the Ministry provided short- 
term funding to these projects through a special 
request to Treasury Board. 

The Ministry is currently reviewing options for 
the most cost-effective ways to fund projects like 
the high-performance computing platforms. A 
report outlining the different options for funding 
and delivery of such projects, with recommenda- 
tions on preferred approaches, was expected for 
fall 2011. In addition, the Ministry reiterated in 
updated program guidelines on its website that 
projects such as the high-performance computing 
platforms are ineligible for Ontario Research Fund/ 
Research Excellence support. 

Research Infrastructure Program 

Recommendation 5 

To ensure that projects funded by the Research 
Infrastructure Program are economically beneficial 
to Ontario, the Ministry of Research and Innovation 
should: 

• only fund projects that are highly aligned with 
Ontario's priorities; and 

• consider funding projects that have not applied 
to, or received funding from, the Canada Foun- 
dation for Innovation, if they offer significant 
benefits to Ontario. 

Status 

In 2009, the Ministry implemented an Ontario 
First approach to funding decisions for the Large 
Infrastructure Program, under which it no longer 
automatically matches CFI investments. Instead, 
funding decisions are now based on a proposal's 
strategic benefits to Ontario and its scientific 
merits. The Ministry will co-fund projects only 
where provincial and federal priorities are aligned. 

The Ministry established five expert strategic 
review panels in various sectors such as health 
sciences and clean technologies to review and 
assess the strategic value of 172 funding requests in 
2009. Each panel consisted of 10 experts from the 
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academic and business communities who in their 
recommendations for project funding considered 
CFI expert-committee reports. To help guide panel 
reviews, the Ministry also provided assessment 
forms requiring panellists to consider key issues 
relating to the strategic value to Ontario of the 
proposals. Panels were asked to group proposals 
into priority categories and on the basis of that 
advice, the Fund's Advisory Board made funding 
recommendations to the Minister, who made the 
final approvals. 

As a result of the new Ontario First approach, 
the Ministry provided $243.3 million in funding to 
priority projects for Ontario in the 2009 infrastruc- 
ture competition. The Ministry also funded four 
projects that did not receive CFI funding but were a 
priority for Ontario, and it chose not to fund seven 
proposals that received CFI assistance because they 
did not rank as high as others in terms of strategic 
benefit to Ontario. 

In addition, the Ministry informed us that it 
formed a working group to provide feedback on 
ways to improve the Ontario First process in 2012 
for future large infrastructure competitions, and to 
continue to ensure that Ontario derives a strategic 
benefit from all research infrastructure projects that 
get provincial funding. 

Colleges and Smaller Institutions 

Recommendation 6 

To ensure that the Ontario Research Fund selection 
process is accessible to all eligible applicants, and 
to help meet the program's overall goal of commer- 
cialization of research, the Ministry of Research and 
Innovation should work with colleges, smaller institu- 
tions, and federal research agencies to ensure that 
the specific requirements and infrastructure needs of 
Ontario colleges and smaller institutions that focus on 
applied research are given appropriate consideration. 

Status 

The Ministry informed us that it communicated 
with all 24 Ontario colleges and with their advo- 



cacy organization as part of its funding selection 
process for the Research Excellence Program. The 
Ministry also launched its College-Industry Innova- 
tion Fund (Innovation Fund) program to provide 
co-funding ($10 million) with a similar CFI pro- 
gram. The Innovation Fund's purpose is to bolster 
the capacity of Ontario colleges to support busi- 
nesses by providing an industry-relevant research 
infrastructure that fosters partnerships with the 
private sector. 

The Ministry said that it invited all colleges to 
information sessions about the Innovation Fund 
competition in 2011, and encouraged them to seek 
assistance from the Ministry for their applica- 
tions. The Ministry also set up a website to inform 
colleges of program details. Notices of intent to 
apply to this fund were to be submitted for the 
first time in June 2011, and 14 colleges submitted 
applications. 

In addition, the Ministry informed us that two 
college representatives were appointed to the 
Ontario First Working Group, alongside two from 
the Ontario Council on University Research and 
two from the Council of Academic Hospitals of 
Ontario. The Working Group provided input on the 
adjudication process to be used in the 2012 Large 
Infrastructure Program competition. 

The Ministry also indicated that it will continue 
to look for ways to strengthen research capacity 
in colleges and smaller institutions. In 2009, for 
example, it committed $10.2 million over three 
years to the Colleges Ontario Network for Industry 
Innovation (Network) to allow it to expand to 20 
colleges from 10. The Network, founded in 2006, 
began as an applied research and development 
network of leading post-secondary institutions 
with a goal to help small- and mid-sized enterprises 
solve technical problems, adapt new technologies 
for the marketplace, and develop new or improved 
products and processes. 

In the most recent round of Ontario Research 
Fund proposals, there were seven funding submis- 
sions from colleges — five for the Large Infrastruc- 
ture Program (October 2008) and two for the 
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Research Excellence Program (October 2010). One 
of these seven submissions was selected, and the 
college received funding for its proposal. 

PROJECT MONITORING 

Research Excellence Program 

Recommendation 7 

To ensure that Research Excellence Program grants 
are used for the purposes intended and that project 
performance is effectively monitored, the Ministry of 
Research and Innovation should: 

• implement a process to identify and follow up 
on projects that are not reporting quarterly as 
required; 

• perform routine, formal monitoring visits 
to verify the information submitted by grant 
recipients, to ensure that program funds are 
being used for the approved research and that 
research milestones have been met; and 

• develop clear guidelines for what independent 
audits are expected to accomplish and report, 
ensure that audit reports are received when due, 
and follow up on issues they identify on a timely 
basis. 

Status 

The Ministry informed us that it established a work- 
ing group for the Research Excellence Program to 
conduct a review of program systems, including 
guidelines, contracting, and project-management 
processes, to improve expenditure management 
and accountability. 

The Ministry also advised us that a Contract 
Management Tool (CMT) has been implemented 
in its research-awards database to assist with the 
management and monitoring of contract compli- 
ance of all Research Excellence projects. The CMT 
was incorporated into the approvals process for 
program reporting in mid-2010 and provides a 
mechanism for collecting financial and perform- 
ance information over the life of a project. This will 
enable quick identification of projects that are not 
meeting contracted reporting dates so that program 



staff can take follow-up action. To date, CMT 
reports have been used to ensure that the Program 
receives quarterly requests for payments from 
recipients, and to track the amounts paid out. 

The Ministry also informed us that it makes site 
visits to funded projects, but there is currently no 
formal process or schedule regarding these visits. 
The development of a formal process is to be dis- 
cussed as part of a business transformation project 
that is currently under way. 

In addition, the Ministry indicated that it con- 
tinues to work with its internal audit department on 
implementing strengthened program-monitoring 
processes, including at least two audits each year 
of selected recipients that receive a large number of 
grants. In May 2011, the Ministry released the first 
such audit report of two funding recipients, includ- 
ing responses from the recipients. The audits found 
that there was generally adequate governance 
over contracts, and that recipients complied with 
contract terms and with the government's transfer- 
payment accountability directive. The audits also 
noted areas for improvement, including program 
monitoring and timeliness of project reporting. 

Research Infrastructure Program 

Recommendation 8 

To more effectively monitor Research Infrastructure 
Program grants and ensure adequate co-ordination 
of oversight processes with the Canada Foundation 
for Innovation (CFI), the Ministry of Research and 
Innovation should: 

• periodically obtain and review the CFI mon- 
itoring reports and audits for selected larger 
Ontario-funded projects to ensure that prov- 
incial funds are being used for their intended 
purpose and funded institutions comply with 
program policies and guidelines; 

• assess the need for ministry staff to conduct site 
visits, especially on the larger projects; and 

• establish a formal agreement with the CFI that 
clearly defines the roles and expectations of each 
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party in the oversight processes for co-funded 
projects. 

Status 

In May 2011, ministry and CFI staff met to discuss 
creation of a formal information-sharing agreement 
for project oversight. They planned to develop an 
agreement to set out the responsibilities of each 
organization in sharing of monitoring, audit, and 
site-visit reports. The Ministry planned to have a 
memorandum of understanding in place with the 
CFI by December 2011, at which time it expected to 
regularly obtain and review audit and monitoring 
reports, and collaborate with the CFI on site visits. 
The Ministry and its internal audit department 
have also been discussing greater Ministry-CFI 
co-operation, including a review of past projects 
audited by the CFI to look for possible gaps in CFI 
monitoring. The Ministry expected to begin receiv- 
ing audit reports from the CFI in fall 2011. 

PROGRAM ADMINISTRATION 

Information Systems 

Recommendation 9 

To ensure that the Ministry of Research and Innova- 
tion has the information needed to effectively oversee 
its Ontario Research Fund program, its information 
system should provide ministry staff with timely 
program-level and project-specific information. 

Status 

The Ministry informed us that it launched the 
Electronic Research and Innovation Management 
System (eRIMS) project in November 2009 to 
implement an electronic grants-management sys- 
tem. The system aims to enhance customer service 
and improve the accountability and transparency 
of the grant application, adjudication, and manage- 
ment process. 

The Ministry indicated that the scope of the 
project includes automation or streamlining of the 
following grant-management processes: 



• application submission, allowing applicants 
to complete and submit application forms 
on-line; 

• application eligibility processing; 

• peer-review management; 

• application adjudication and selection; 

• timely communication of key decisions; 

• contracts and payment authorizations; 

• disbursement and report-back requirements; 
and 

• project management , including project 
budgeting, accounting, information manage- 
ment, and performance measurement. 

The Ministry informed us that it selected the 
Premier's Discovery Awards program as the pilot 
program for eRIMS to test the processes using 
transactions from an actual project. According to 
the Ministry, some core functionality issues discov- 
ered during the pilot have been resolved and as of 
August 2011, the application was in the final testing 
stage. After the completion of testing, the Ministry 
will roll out the system for its Premier's Discovery 
Awards program, with more programs to be added 
shortly thereafter. 

Private-sector Partner Contributions 

Recommendation 10 

To provide assurance that in-kind private-sector con- 
tributions are fairly valued, the Ministry of Research 
and Innovation should: 

• ensure that grant recipients comply with the 
policies adopted for the program to assess the 
value of in-kind contributions; and 

• periodically verify that independent valuations 
of substantial in-kind contributions have been 
performed to support values reported by grant 
recipients. 

Status 

The Ministry advised us that it removed from 
the Research Excellence Program guidelines a 
reference to CFI policy on the valuation of in-kind 
contributions. It also published more specific 
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guidelines on how to determine the value of some 
eligible in-kind contributions, and defined others 
that are ineligible. 

The Ministry informed us that it continues to 
assess in-kind contributions from private-sector 
partners during the contracting process, as it did at 
the time of our 2009 audit. However, it has stated 
more clearly in its latest guidelines the level of 
detail required in supporting documents. 

With regard to the Research Infrastructure 
Program, the Ministry continues to rely on CFI 
due diligence to ensure that reliable valuations 
are done in accordance with the federal guide on 
the audit of contributions. Although it has not 
reviewed CFI work in this area, the Ministry was 
working with the federal organization on a new 
agreement about verifications. 



After consulting stakeholders, the Ministry 
concluded that requiring third-party verifications 
would be unreasonable, given that it can be difficult 
to find the appropriate expertise and that obtaining 
such verifications is often prohibitively expensive 
relative to the funding provided. 

Instead, the Ministry relies on the institutions 
and their private-sector partners to justify how 
they determined the value of in-kind contribu- 
tions. The Ministry has indicated that it requires 
the institutions to provide support to justify the 
value of all in-kind contributions, regardless of the 
amount. An institution must, for example, attest 
to the fact that the valuations for services from its 
staff are based on actual salaries and benefits of 
those staff. 
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■ Section 

Ontario Works Program 

Follow-up on VFM Section 3.11, 2009 Annual Report 



Background 



Under the Ontario Works Act, 1997, the Ministry 
of Community and Social Services (Ministry) 
provides income and employment assistance to 
260,000 individuals who are unemployed, along 
with their qualifying family members for a total 
of approximately 474,000 people. The income 
assistance is intended to help recipients with basic 
living expenses such as food, clothing, and shelter, 
while the employment assistance includes a variety 
of activities designed to increase employability 
and help recipients obtain paid employment in 
order to become self-reliant. For the 2010/11 fiscal 
year, the Ministry's Ontario Works expenditures 
totalled over $2.4 billion — $2 billion for income 
assistance, $189 million for employment assist- 
ance, and $247 million for program administra- 
tion ($1.9 billion in 2008/09— $1.5 billion for 
income assistance, $171 million for employ- 
ment assistance, and $194 million for program 
administration). 

The Ontario Works program is delivered on 
behalf of the Ministry by 47 Consolidated Munici- 
pal Service Managers and District Social Services 
Administration Boards as well as 101 First Nations, 
all referred to as service managers. A service 
manager is typically either a large municipality or a 
grouping of smaller ones, and each is accountable 
to one of the Ministry's nine regional offices. The 
Ministry and the service managers share the total 



financial and employment assistance costs of the 
Ontario Works program. The Ministry, which cur- 
rently pays 81% of these costs, has committed to 
start gradually increasing its share in 2010 until it 
pays 100% in 2018. Administrative costs will con- 
tinue to be shared on a 50/50 basis. 

In our 2009 Annual Report, we noted that 
although the Ministry had implemented a number 
of the recommendations contained in our last audit 
of the program in 2002, there had been limited 
improvement in the overall administration of the 
program since that time. It remained our view that 
the Ministry still had inadequate oversight and 
assurance that only eligible individuals received the 
correct amount of financial assistance. 

Our more significant concerns about the Min- 
istry's oversight of Ontario Works program delivery 
by the service managers, noted in our 2009 Annual 
Report, included the following: 

• During the Ontario Works application pro- 
cess, service managers relied on individuals 
to provide almost all of the information used 
to assess their eligibility for assistance and 
seldom undertook the required third-party 
verifications designed to help assess the 
completeness and accuracy of the information 
provided by applicants. 

• Benefits for such things as community and 
employment start-up activities were often 
paid without any evidence that the activity 
had occurred and in amounts that exceeded 
the established maximums. 
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• The total amount spent on the special dietary 
allowances had increased from $5 million in 
the 2002/03 fiscal year to more than $67 mil- 
lion during 2008/09, and we noted that many 
special dietary allowances were being paid 
under questionable circumstances. 

• Unrecovered overpayments to about 350,000 
current and former Ontario Works recipients 
had increased 45%, from $414 million in 
February 2002 to $600 million as of March 31, 
2009. Efforts by service managers to recover 
these overpayments had been minimal, pos- 
sibly owing to the lack of financial incentive 
for them to do so. 

• Many tips from the fraud hotline had been 
either inadequately investigated or ignored. 

• The Ministry did not have enough informa- 
tion to assess whether employment assistance 
funds were being used as intended and were 
helping people obtain employment. 

• The Ministry's examination of a sample of 
service managers' reimbursement claims 
for the Ministry's share of program costs did 
not occur annually as required, nor did the 
examinations ensure that submitted claims 
were complete, accurate, and based on actual 
payments made to assistance recipients. 

Despite improvements to the Ministry's Service 
Delivery Model Technology (SDMT) information 
system, which has been used by service managers 
to deliver the Ontario Works program since 2002, 
we continued to have concerns about the system's 
reliability and its known deficiencies. 



Status of Actions Taken on 
Recommendations 



In our 2009 report on the Ontario Works program, 
the Ministry agreed with all of our recommenda- 
tions, and in its responses to our recommendations 
indicated that it was going to take the necessary 



corrective actions. At the time of our follow-up, the 
Ministry had several initiatives under way, such as a 
new monitoring framework to assist with program 
oversight and compliance with program require- 
ments, a new IT system, and a new system to priori- 
tize high-risk cases for review to help ensure that 
only eligible recipients continue to receive assist- 
ance. However, it will take more time before the 
initiatives can be fully implemented. In addition, 
in November 2010, the government announced 
the launch of a major review of the social assist- 
ance programs in Ontario. However, insufficient 
monitoring of service managers remains a concern. 
Consequently, the Ministry does not yet have 
adequate assurance that, for example, recipients 
are financially eligible for Ontario Works benefits, 
either initially or on an ongoing basis, and that pay- 
ments are being made in the correct amount. The 
Ministry has made a significant effort to enhance 
controls over the special dietary allowance and 
improve processes for the recovery of overpay- 
ments, and has implemented a new funding model. 

The status of actions taken on each of our rec- 
ommendations at the time of our follow-up was as 
follows. 

MINISTRY OVERSIGHT AND CONTROL OF 
PROGRAM DELIVERY 

Initial Financial Eligibility Assessment 

Recommendation 1 

To ensure that an individual's initial financial eligibil- 
ity for Ontario Works benefits is adequately deter- 
mined and that the correct amount of assistance is 
paid, the Ministry of Community and Social Services 
should make certain that Consolidated Municipal 
Service Managers: 

• visually verify documents or obtain copies of all 
documents required to establish an individual's 
identity and legal status in Canada, especially 
Social Insurance Number cards; and 

• comply in all cases with the requirement to 
verify an applicant's declared income and 
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assets with the third parties who have entered 
into information sharing agreements with the 
Ministry. 

Status 

The Ministry had made limited progress in address- 
ing this recommendation at the time of our follow- 
up. However, over the longer term, the Ministry 
informed us that it expects to improve its oversight 
of service managers by developing a new, more 
effective social assistance computer system, to be 
implemented by April 2013, and by introducing 
a risk-based monitoring framework, which is 
expected to be rolled out in April 2012. 

As an interim strategy, the Ministry conducted 
compliance reviews using a risk-based approach at 
all Ontario Works delivery sites during the first half 
of 2011 to evaluate compliance with legislation, 
regulations, and policy directives. These reviews 
targeted the following high-risk program areas, 
selected on the basis of the findings in our 2009 
report: intake and application, overpayments, 
discretionary benefits provided to recipients, and 
participation agreements. However, the outcome of 
these reviews was not available at the time of our 
follow-up. 

The Ministry also told us that it has provided 
optional training to service manager staff on the 
Equifax credit reports — a third-party verifica- 
tion procedure — to assist them with reading and 
understanding the reports. In addition, the Ministry 
amended the Ontario Works policy directives to 
strengthen its requirements regarding third-party 
verifications. 

Financial Eligibility Reassessments 

Recommendation 2 

To ensure that recipients continue to be financially 
eligible for Ontario Works benefits and to avoid 
overpayments, the Ministry of Community and Social 
Services should make certain that Consolidated Muni- 
cipal Service Managers: 



• complete financial reassessments on each recipi- 
ent at least once every 12 months as required; 

• use the Ministry-prescribed checklist when 
conducting a financial reassessment and obtain 
sufficient documentation, including third-party 
verifications, to support the outcome of the 
review; and 

• help ensure that the risk flags in the Service 
Delivery Model Technology system are effective 
and are used to prioritize high-risk cases for 
review. 

Status 

As already noted, the Ministry advised us that it is 
developing a new, more effective computer system, 
to be implemented by April 2013, and a risk-based 
monitoring framework to strengthen program mon- 
itoring and compliance with legislation, regula- 
tions, and policy directives, which is expected to be 
rolled out in April 2012. When implemented, these 
new initiatives are expected to help identify areas of 
non-compliance with program requirements, which 
will provide a basis for taking corrective action. 
Until then, the Ministry has no assurance that 
service managers are completing financial reassess- 
ments on each recipient at least every 24 months as 
currently required or that they are using the Min- 
istry-prescribed checklist and obtaining sufficient 
documentation, including third-party verifications, 
in order to support the outcome of the review. 

With regard to our recommendation on pri- 
oritizing high-risk cases for review, the Ministry 
informed us that it has developed a new risk-based 
eligibility reassessment process in conjunction with 
Equifax Canada, called the Eligibility Verification 
Model. This new model is expected to assist in the 
identification and prioritization of high-risk cases 
for eligibility reviews by linking Ontario Works data 
with Equifax consumer credit databases. Testing 
of the new process began in October 2010, but it 
had not yet been implemented at the time of our 
follow-up. 
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Other Income Reporting 

Recommendation 3 

To ensure that financial assistance provided by 
Ontario Works is in the correct amount and to mini- 
mize overpayments, the Ministry of Community and 
Social Services should make certain that Consolidated 
Municipal Service Managers receive a monthly 
income report from each recipient, unless they waived 
the requirement for sound reasons that are docu- 
mented on file. If it is the Ministry's intention that 
Consolidated Municipal Service Managers require the 
report on an exception basis only, that should be more 
clearly communicated. 

Status 

The Ministry informed us that it has reviewed and 
revised its income reporting directive to clarify and 
provide examples of conditions where the monthly 
reporting requirement can be waived, and to 
strengthen the documentation requirements to now 
include current income at the time of the waiver, 
the length of time the waiver is to be in place, and 
the need to review the waiver on a regular basis. 
However, without conducting any follow-up or 
verification reviews at the service managers, these 
revisions, in themselves, do not provide assurance 
to the Ministry that the required monthly income 
reports are now being received on a consistent 
basis. 

Other Financial Assistance and Benefits, 
and Special Dietary Allowance 

Recommendation 4 

To ensure that supplemental financial assistance and 
benefits provided under the Ontario Works program 
are reasonable and appropriate, the Ministry of Com- 
munity and Social Services should make certain that 
Consolidated Municipal Service Managers: 

• comply with the requirement to document the 
need and eligibility for supplemental financial 
assistance and benefits, and provide such 
assistance and benefits within the established 
maximum amounts; and 



• obtain the required documentation to assess 
and substantiate the reasonableness of costs 
reimbursed. 

In addition, the Ministry should review the special 
dietary allowance with a view to limiting its possible 
abuse. 

Status 

As mentioned previously, as a long-term strategy, 
the Ministry expects that the new social assist- 
ance computer system and risk-based monitor- 
ing framework it is developing will strengthen 
program monitoring of legislative, regulatory, and 
policy requirements. In the interim, the Ministry 
completed compliance reviews targeting high-risk 
program areas identified in our 2009 report, includ- 
ing supplementary benefits provided to recipients. 
However, the outcome of these reviews was not 
available at the time of our follow-up. 

With regard to the special dietary allowance, the 
Ministry undertook a forensic audit to determine 
the extent of possible misuse of the allowance, 
which corroborated many of the findings in our 
2009 Annual Report. In March 2010, the govern- 
ment announced plans to eliminate the special 
dietary allowance and create a new medically 
based nutritional supplement program for social 
assistance recipients with severe medical needs that 
would be administered by the Ministry of Health 
and Long-Term Care. However, in November 2010, 
the government announced that the special dietary 
allowance would continue, but would be revised to 
comply with a previous order of the Human Rights 
Tribunal of Ontario and with the recommendations 
of an expert committee. 

The changes to the administration of the special 
dietary allowance, which took effect in April 2011, 
included the following: 

• removing from the list of eligible conditions 
those that the expert committee found to not 
require a special dietary allowance; 

• revising the application form to require 
recipients to consent to the release of relevant 
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medical information by their physician to sup- 
port their application; 

• requiring Ontario Works recipients to reapply 
for the special dietary allowance, which 

has resulted in a drop of about 14,500 cases 
receiving the allowance, or a funding impact 
of about $2.6 million per month; 

• filing complaints with the College of Phys- 
icians and Surgeons where deemed appropri- 
ate; and 

• confirming that service managers and their 
staff have the legislative authority to deter- 
mine eligibility for the allowance, including 
the authority to request additional informa- 
tion or deny an application in cases where the 
information provided is believed to be false or 
incorrect. 

In addition, the Ministry began to use its IT sys- 
tem to help identify questionable trends in a timely 
manner so that appropriate action could be taken 
by service managers. 

Although these changes are intended to improve 
the administration of the special dietary allowance, 
the Ministry will still need to monitor the service 
managers to ensure that they are complying with 
the new requirements in order to limit potential 
abuse of the allowance. 

Overpayments 

Recommendation 5 

To better utilize its limited resources and maximize 
the recovery of previous overpayments, the Ministry 
of Community and Social Services should: 

• ensure that Consolidated Municipal Service 
Managers assess the collectibility of all out- 
standing overpayments — particularly those 
designated as temporarily uncollectible — and, 
where warranted, recommend that the overpay- 
ments be written off so that more focus can be 
placed on those accounts where collection efforts 
are more apt to yield results; and 

• evaluate the merits of the 2006 pilot project that 
transferred some overpayments to the Ministry's 



Overpayment Recovery Unit and, if necessary, 
consider implementing other alternatives for 
bringing a more intensive and focused collection 
effort to bear on those inactive accounts that 
have a greater likelihood of collection. 

Status 

In February 2010, the Ministry established the 
Social Assistance Overpayment Recovery Working 
Group to thoroughly review its overpayment poli- 
cies and recovery practices, to research industry 
standards, and to develop strategies for improving 
the Ministry's collection efforts. This group also 
reviewed the 2006 pilot project involving the Min- 
istry's Overpayment Recovery Unit (ORU) . 

The working group issued a report in December 
2010 and an implementation plan was developed 
for, among other things, the writeoff of uncollect- 
ible overpayments. A review was then undertaken 
of existing overpayments and, as a result, certain 
overpayments deemed to be uncollectible were 
recommended for writeoff. At the time of our 
follow-up, the majority (about $80 million worth) 
of these uncollectible accounts had been written 
off. The writeoff of these uncollectible accounts 
should allow the Ministry and service managers to 
better focus their collection efforts on accounts that 
have a better chance of being collected. 

The working group also reviewed the operations 
of the ORU in the 2006 pilot project and the results 
it achieved, and recommended that the program 
be expanded to all service managers, since the 
amounts collected exceeded the cost of the collec- 
tion efforts. It also recommended improvements 
aimed at achieving more effective and cost-efficient 
results, such as having a common IT system for the 
service managers and the ORU, and conducting effi- 
ciency reviews at both offices to avoid duplication. 

Potentially Fraudulent Claims 

Recommendation 6 

To ensure that only eligible individuals receive finan- 
cial assistance and that adequate action is taken 
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when suspected fraud is reported, the Ministry of 
Community and Social Services should ensure that 
Consolidated Municipal Service Managers: 

• in a timely manner, follow up on all fraud tips 
and investigate those that appear to be legitim- 
ate; and 

• where the investigation indicates that a poten- 
tial fraud has occurred, provide sufficient 
evidence to justice authorities to enable them to 
pursue prosecution of the perpetrators. 

Status 

The Ministry has made limited progress in address- 
ing this recommendation. It advised us that, as 
a first step, it undertook a review to identify best 
practices in fraud prevention and detection from 
other jurisdictions and from its service managers 
across Ontario, with recommendations being 
expected in fall 2011. 

For cases where an investigation indicates that a 
potential fraud has occurred, the Ministry is in the 
process of negotiating a provincial protocol with 
the Ontario Provincial Police that, when completed, 
can be used as a model for service managers in 
developing protocols for referring potential fraud 
cases to their own local policing authorities. 



determining the most appropriate activities to 
help recipients transition to financial independ- 
ence; and 

• that the Ministry review the reasonableness 
of service managers' allowing — often for pro- 
longed periods of time — independent job-search 
activities as the primary employment assistance 
activity to nearly two-thirds of all recipients. 

Status 

The Ministry has not made significant progress 
to date in addressing this recommendation. It 
conducted compliance reviews during the first half 
of 2011 at all Ontario Works delivery sites to evalu- 
ate compliance with legislation, regulations, and 
policy directives, including participation agreement 
requirements. However, the results of these reviews 
had not been summarized or reported on at the 
time of our follow-up. 

In addition, the Ministry had not yet addressed 
our recommendation to ensure that caseworkers 
assess the skills and experience of recipients to help 
determine the most appropriate activities for them, 
nor had it reviewed the reasonableness of letting 
applicants conduct independent job-search activ- 
ities over prolonged periods. 



Participation Agreements 



Tasks 



Recommendation 7 

To ensure that the Ontario Works program is effective 
in transitioning recipients to paid employment and 
self-reliance, the Ministry of Community and Social 
Services should monitor Consolidated Municipal 
Service Managers to make certain: 

• that participation agreements are on file for 
all Ontario Works recipients and that each 
agreement is reviewed and updated every three 
months as required; 

• that the reasons for deferring participation 
agreement requirements are adequately sup- 
ported and documented on file; 

• that caseworkers assess recipients' skills and 
experience, and document caseworker input in 



Recommendation 8 

To ensure that Ontario Works benefits continue to 
be paid only to eligible individuals and in the correct 
amount, the Ministry of Community and Social Ser- 
vices should monitor whether Consolidated Municipal 
Service Managers are making reasonable efforts to 
address all system-identified tasks that require action 
or follow-up. 

Status 

The Ministry advised us that it undertook a 
cleanup exercise for outstanding tasks in its com- 
puter system whereby it removed 40% of all open 
tasks from the system after determining that they 
were redundant. With regard to new tasks being 
created, we were also advised that the Ministry 
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has simplified the programming so that tasks 
considered unnecessary are no longer generated 
automatically. However, the Ministry does not 
currently monitor service managers to determine 
that they are making reasonable efforts to address 
all system-identified tasks that require action or 
follow-up. 

Ministry Monitoring of Consolidated 
Municipal Service Managers 

Recommendation 9 

To ensure that subsidy claims are reimbursed in 
the correct amount based on reliable information 
provided by the Consolidated Municipal Service Man- 
agers, the Ministry of Community and Social Services 
should: 

• conduct at least one subsidy claims examination 
per service manager annually as required and 
do so on a timely basis; 

• make certain that work conducted during 
subsidy claims examinations is adequately 
completed and demonstrates whether the claim 
is based on complete and accurate information 
about payments to assistance recipients; and 

• make certain that adequate supporting docu- 
mentation is submitted by the service managers 
and reviewed by the Ministry prior to payment. 

Status 

The Ministry no longer conducts subsidy claims 
examinations but is currently developing a risk- 
based framework to help strengthen program mon- 
itoring and compliance with legislation, regulations, 
and policy directives, which is expected to be rolled 
out in April 2012. As part of the new framework, the 
Ministry plans to use operational reports to monitor 
program delivery, which will include data related 
to financial expenditures and subsidy claims. These 
reports will also be used as inputs to the Ministry's 
risk assessment to determine whether intervention 
or corrective action is required. 

To assist in making certain that adequate 
supporting documentation is submitted by the 



service managers and reviewed by the Ministry 
prior to payment, the Ministry was in the process 
of developing two new guides — a Subsidy Claim 
Preparation Guide to assist service managers and 
a Subsidy Claims Approval and Review Guide for 
ministry staff. These guides were still in draft for- 
mat at the time we were conducting our follow-up. 

Program Administration Costs 

Recommendation 10 

To ensure that Ontario Works administration is 
funded equitably across the province, the Ministry of 
Community and Social Services should: 

• establish more needs-based funding of adminis- 
trative costs that reflects variations in caseloads; 
and 

• obtain better information about actual adminis- 
trative costs being incurred. 

Status 

The Ministry informed us that in April 2011 it 
implemented a new funding model for Ontario 
Works with the goal of providing a responsive and 
equitable approach to funding. This new approach 
combines Ontario Works administration and 
employment assistance funding into one allocation, 
giving service managers the flexibility to determine 
how best to allocate funding for all aspects of pro- 
gram delivery while simplifying financial reporting. 
Funding provided to service managers is based on 
an amount of $2,016 per case. 

We were advised that this new funding must 
be spent on eligible program delivery costs as 
outlined by the Ministry, such as staffing, benefits, 
travel, and the purchase of services for employ- 
ment assistance activities. In order to help ensure 
that funds are spent as required, service managers 
must report quarterly to the Ministry on how 
the funds were spent, with the amounts broken 
down by the type of eligible expense. A portion of 
this funding may be recovered by the Ministry if 
it is spent on ineligible activities or if it does not 
achieve expected program outcomes. 
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Employment Assistance Costs 

Recommendation 11 

To ensure that employment services are effective in 
helping recipients find employment and represent 
value for money spent, the Ministry of Community 
and Social Services should: 

• assess the effectiveness of the various types of 
employment assistance being offered by each 
Consolidated Municipal Service Manager, 
particularly the independent job search when 
recipients are assigned to it for long periods of 
time; and 

• make certain that all employment assistance 
funding is spent prudently and for the intended 
purpose. 

Status 

The Ministry has not yet assessed the effectiveness 
of the various types of employment assistance 
offered by the service managers, including the 
appropriateness of independent job search. 

As mentioned previously, the Ministry imple- 
mented a new funding approach for Ontario Works 
in April 2011 in which a portion of funding is based 
on employment outcomes. It combines Ontario 
Works administration and employment assist- 
ance funding into one allocation, giving service 
managers the flexibility to determine how best to 
allocate funding for program delivery. Service man- 
agers are now required to report detailed informa- 
tion on their eligible expenditures to the Ministry 
on a quarterly basis, as opposed to providing it only 
at year-end, as in the past. 

Measuring the Performance of the 
Ontario Works Program and Consolidated 
Municipal Service Managers 

Recommendation 12 

The Ministry of Community and Social Services 
should build on its planned results-assessment for 
employment assistance funding by developing per- 
formance measures that will enable it to evaluate the 



effectiveness of the administration of the much larger 
income assistance aspect of Ontario Works over time. 

Status 

The Ministry has not yet identified specific per- 
formance measures to enable it to evaluate the 
effectiveness of the program's income assistance 
component, which accounts for more than 80% of 
total program costs. 

It has, however, developed a quarterly Oper- 
ational Indicators Report for the service managers 
to report relevant information to support oper- 
ational management and decision-making. The 
report provides a snapshot of the health of social 
assistance delivery through the use of specific indi- 
cators for workload, case management, customer 
service, accountability, and finance. The goals of 
collecting this information are to support early 
detection of problems or operational issues, provide 
context for perceived trends and anomalies, and 
identify areas that need more detailed monitoring. 

SERVICE DELIVERY MODEL 
TECHNOLOGY SYSTEM 

Recommendation 13 

To ensure that Consolidated Municipal Service 
Managers can rely on systems and reports to produce 
proper payments, and accurately record and manage 
information regarding those payments, the Ministry 
of Community and Social Services should address the 
Service Delivery Model Technology system deficiencies 
noted in this report, including those that prevent ser- 
vice manager staff from having the information they 
need to effectively manage program expenditures. 

Status 

In November 2009, the government approved the 
Social Service Solution Modernization Project, 
which will replace the current system. This new 
application is intended to provide greater flexibility 
to respond to policy and program changes, support 
effective service delivery, and enhance audit and 
controllership capability. The total cost of the 
new system is estimated to be $165 million, plus 



2011 Annual Report of the Office of the Auditor General of Ontario 



maintenance costs of $37 million up to the imple- 
mentation date. 

According to the Ministry, the project was on 
track at the time of our follow-up and will be imple- 
mented in two phases: 

• The first phase is an on-line application 
for social assistance that was implemented 
province-wide, excluding the City of Toronto 
and First Nations communities, in May 2011. 
The Ministry informed us that the City of 
Toronto would continue to use its own on-line 
application until October 2011, at which time 
it would adopt the provincial system. 

• The second phase is the replacement of the 
current system, which will be implemented in 
spring 2013. 




Chapter 4 



Ministry of Municipal Affairs and Housing 



Section 



Social Housing 




Follow-up on VFM Section 3.12, 2009 Annual Report 



Background 



Social housing is rental accommodation developed 
with government assistance for a range of low- and 
moderate-income households, including families 
with children, couples, singles, and seniors. It can 
be owned by governments, as in the case of public 
housing, or by non-profit or co-operative organiza- 
tions. In Ontario, households in social housing that 
receive a subsidy to help pay rent typically pay a 
maximum rent equal to about 30% of their total 
pre-tax income. 

Most social housing in Ontario was built between 
the mid-1960s and the mid-1990s by the Canada 
Mortgage and Housing Corporation (CMHC) and 
the provincial government. In December 2000, 
the province passed the Social Housing Reform Act, 
2000, which required municipalities to assume 
responsibility for social-housing programs previ- 
ously administered by the CMHC and the province. 
The province designated 47 regional Consolidated 
Municipal Service Managers (Service Managers) 
to administer social-housing programs at the local 
level. At the end of the 2010/11 fiscal year, there 
were about 260,000 units of social housing in 
Ontario, consisting of 100,000 public-housing units 
and 160,000 non-profit and co-operative units, 
essentially the same numbers as at the time of our 
2009 Annual Report. 



Both from a value-for-money perspective and 
from the perspective of those who live there, it is 
critical that social housing be maintained in good 
condition. As well, sufficient and affordable social 
housing can also have a significant impact on the 
health and safety of those Ontarians who depend 
on subsidized housing for a place to call home. 
However, in our 2009 Annual Report we reported 
that the Ministry of Municipal Affairs and Housing 
(Ministry) collected little information on how well 
the $40 billion in social-housing stock was being 
maintained or whether there was an adequate sup- 
ply to meet local needs. Our observations included: 

• As of December 31, 2008, the number of 
households on waiting lists for social housing 
across the province totalled about 137,000. 
In many urban centres, the average wait time 
to secure accommodation was more than five 
years — and one municipality had reported a 
wait time of 21 years for all categories of ten- 
ants except seniors. 

• The deteriorating condition of social -housing 
stock — particularly the public-housing port- 
folio, whose units were an average of 40 years 
old — had been a significant and growing con- 
cern for municipalities. In 2006, for instance, 
the Toronto Community Housing Corporation 
identified immediate capital-repair needs of 
$300 million for its 60,000 public -housing 
units. However, the Ministry had no up-to- 
date and reliable information on the overall 
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condition of the social-housing stock on a 
province-wide basis. 

• A large number of the federal government's 
funding agreements with housing providers 
would start to expire in 2015, with no guaran- 
tee that they would be renewed. Without con- 
tinued funding, some existing social-housing 
projects would not be financially viable, 
even though Service Managers would still be 
required by law to maintain the prescribed 
minimum number of housing units. The Min- 
istry had no firm plans to address concerns 
regarding this possible ending of federal 
funding. 

• In partnership with the federal government, 
Ontario had in recent years provided Service 
Managers with some additional funding for 
new housing programs. There was a general 
lack of reporting on the success of these 
programs. For example, although one such 
program increased the supply of housing, the 
stipulated rent to be charged for more than 
half the units would not be considered afford- 
able for households on, or eligible to be on, 
waiting lists. Better reporting and oversight 
was needed to ensure that these stimulus 
investments are spent cost-effectively and 
achieve the desired results. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 



Status of Actions Taken o 
Recommendations 



On the basis of information provided by the Min- 
istry, we concluded that it had made some progress 
on all of our recommendations, with significant 
progress being made on Recommendation 1. The 
status of action taken on each of our 2009 recom- 



mendations at the time of our follow-up was as 
follows. 



PROVINCIAL STRATEGY ON SOCIAL 
HOUSING 

Recommendation 1 

To better define and fulfill the province's roles for 
ensuring sustainable, well-maintained social housing, 
the Ministry of Municipal Affairs and Housing should: 

• establish a comprehensive strategic plan that 
includes measurable goals and performance 
outcomes; 

• work with municipalities to ensure a co-ordinated 
and integrated housing strategy within the 
province, and gather the information necessary to 
monitor progress on the strategy and on the goals 
and outcomes established; and 

• consider requiring all Consolidated Municipal 
Service Managers to develop local strategic 
plans, and encourage the sharing of best practi- 
ces in developing such plans. 

Status 

In our 2009 Annual Report, we found that despite 
the significant change in the responsibilities for 
delivery of social housing, there was no provincial 
strategy to ensure the continued provision of suf- 
ficient and well-maintained housing. We also found 
at the time that the Ministry's latest Results-based 
Plan, a document that all Ontario government 
ministries are required to submit to help ensure 
that their programs achieve the desired outcomes, 
lacked measurable outcomes for success. 

The Ministry agreed with our recommendation 
and indicated in its response to our 2009 Annual 
Report that it had then completed "over 13" public 
consultations with key stakeholders across the prov- 
ince to initiate the development of a comprehensive 
housing strategy and to develop social-housing 
performance measures that all municipalities would 
be required to report on annually. 

The Ministry released its new Long-Term Afford- 
able Housing Strategy (Strategy) in November 2010. 



The Strategy included passing legislation that 
would support a community-centred approach with 
increased flexibility for adapting to the different 
needs of local communities and would simplify the 
rent-geared-to-income calculation process. 

The Strategy also included a commitment to 
work with the Canada Revenue Agency to create an 
automated income-tax-based system for determin- 
ing the income of social-housing applicants and 
tenants. The Ministry further informed us that 
the Strategy would be supported by a number of 
performance measures to track progress, including 
housing measures, affordability indicators, tenant 
satisfaction surveys, and Service Manager measures 
to track progress in meeting local needs. 

Although draft social- and affordable-housing 
indicators had been developed for the Ministry's 
Municipal Performance Measure Program at the 
time of our follow-up, these measures were still 
being evaluated to ensure alignment with the new 
Strategy and to minimize duplication. The Ministry 
also acknowledged that several other measures 
were still under development, and that it would 
take some time working with Service Managers to 
complete this exercise. 

The Strong Communities through Affordable 
Housing Act, 2011 (Act) received Royal Assent in 
May 2011, and most components of the Act will 
come into force on January 1, 2012. The Ministry 
informed us that once the Act takes effect, local 
Service Managers will be required to develop local 
housing and homelessness plans that address local 
community priorities and better target housing 
resources to people in need. The Ministry also 
indicated that it had developed a new framework 
for how housing and homelessness plans should 
support local communities. A regulation under 
the Act requires that such plans be in place by 
January 1, 2014. 
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SUFFICIENT AND WELL-MAINTAINED 
SOCIAL HOUSING 

Recommendation 2 

To help provide sufficient social housing efficiently 
and make the most of available funding, the Ministry 
of Municipal Affairs and Housing should work with 
Consolidated Municipal Service Managers to: 

• establish more comprehensive reporting of infor- 
mation on social-housing portfolios and wait 
times so this can be taken into consideration 

in addressing the housing needs of individual 
municipalities; 

• identify ways to better and more equitably 
address the issue of lengthy wait times in many 
municipalities; and 

• better co-ordinate housing and other support 
services with other provincial and municipal 
stakeholders. 

Status 

In our 2009 Annual Report, we noted that since the 
devolution of housing responsibilities to the muni- 
cipal level, ministry oversight activities had been 
minimal, and that the Ministry had little informa- 
tion on the often lengthy wait times for social hous- 
ing, local vacancy rates, or details regarding the 
condition of the housing stock. We also noted that 
there were three provincial ministries that admin- 
ister more than 20 housing and related programs, 
and that co-ordination among these programs was 
often lacking, resulting in a fragmented and often 
inefficient approach to meeting client needs. 

The Ministry indicated in its response to our 
2009 Annual Report that it agreed with our recom- 
mendation and would consider it in the develop- 
ment of a new housing strategy. The Ministry also 
informed us that it would work with municipalities 
to identify other areas where additional and consist- 
ent information was available and could be of use 
to the Ministry. Furthermore, the Ministry advised 
us at the time that it would work to develop a con- 
solidated housing service that better co-ordinated 
its housing and other support services with other 
provincial and municipal stakeholders. 
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During our follow-up, the Ministry informed us 
that once the Strong Communities through Afford- 
able Housing Act takes effect, each Service Manager 
will be required to establish a tenant selection 
system. The Act provides flexibility for Service 
Managers to adopt alternatives to the currently pre- 
dominant first-come, first-served approach. In Feb- 
ruary 2011, when the Ministry hosted a stakeholder 
session exploring alternative selection systems, 
some stakeholders suggested new information and 
reporting requirements to support such alterna- 
tives. The Ministry advised us that it is considering 
these stakeholder suggestions. 

Recommendation 3 

To ensure that the housing stock is safe and of accept- 
able quality and that it will achieve its expected 
service life, the Ministry of Municipal Affairs and 
Housing should work with Consolidated Municipal 
Service Managers to: 

• carry out periodic building-condition assess- 
ments and ensure that such information is 
summarized on a province-wide basis; and 

• develop an effective funding and financing 
strategy for raising the capital investment 
required to reduce the capital maintenance 
backlog and sustain proper maintenance of 
housing stock, including consideration of 
requirements that a capital reserve be estab- 
lished for public-housing stock. 

The Ministry should also continue to work 
with the Social Housing Services Corporation to 
assess the cost/benefit of implementing modern 
energy-efficient measures, and facilitate adoption 
of such measures by housing providers. 

Status 

In our 2009 Annual Report, we noted that the 
Ministry did not have up-to-date and reliable infor- 
mation on the condition of the province's social- 
housing stock or on the maintenance and asset 
management practices of its Service Managers. 
We further noted that the condition of the housing 
stock had deteriorated over the decade since the 
devolution of responsibilities to municipalities. 



In its response to our 2009 Annual Report, the 
Ministry indicated that it had helped establish an 
Asset Management Centre for Excellence in 2008 to 
provide support and expertise that social-housing 
providers could draw upon in maintaining their 
buildings. It also indicated that most social-housing 
providers could apply to Infrastructure Ontario for 
low-cost capital loans under the government's 2008 
Poverty Reduction Strategy. With respect to imple- 
menting energy-efficient measures, the Ministry 
further informed us at the time that $70 million 
was targeted for renewable-energy initiatives under 
a new federal-provincial $704 million Social Hous- 
ing Renovation and Retrofit Program. 

At the time of our follow-up, the Ministry 
informed us that its Asset Leveraging Working 
Group was still considering the feasibility and 
merit of a number of proposals for refinancing 
and renewing the province's deteriorating social- 
housing portfolio. In the meantime, the Ministry 
had allocated $352 million in the 2009/10 fiscal 
year and another $352 million in 2010/11 for 
renovation and retrofit work. It indicated that these 
funds, combined with a $100-million allocation for 
capital investment in the 2008/09 fiscal year, were 
intended to have a significant impact on the capital- 
repair backlog for social housing. The Ministry also 
updated us on the status of the renewable-energy 
initiatives, indicating that almost $73 million had 
been committed for installation of solar voltaic, 
geothermal, and solar thermal systems. 

FEDERAL FUNDING OF SOCIAL HOUSING 

Recommendation 4 

To mitigate the possible impact of continuing 
decreases in federal funding on the supply of social 
housing, the Ministry of Municipal Affairs and Hous- 
ing should: 

• develop a plan for options, should negotiations 
with the federal government for continued 
funding for the social-housing portfolio be 
unsuccessful; 
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• work with Consolidated Municipal Service 
Managers on alternatives to the current sys- 
tem of maintaining the required number of 
housing units with an aim to better match the 
supply of social housing to the demand in each 
municipality; 

• review its current methodology to ensure fund- 
ing allocations are fair and federal funds are 
spent on eligible housing programs; and 

• provide a full and public accounting of how all 
federal funding provided for social housing was 
spent. 

Status 

In our 2009 Annual Report, we noted that federal 
government operating agreements with housing 
providers had begun expiring, with a large number 
of agreements set for expiry starting in 2015. The 
federal government at the time provided (and 
continues to provide) most of the funds for social 
housing, and has made no commitment to renew 
this funding as the agreements expired. We further 
noted that the Ministry had no contingency plan for 
addressing this issue, and that under the province's 
Social Housing Reform Act, 2000, Service Managers 
were required to maintain a prescribed minimum 
number of rent-geared-to-income units regardless 
of funding. Some Service Managers had also voiced 
concerns that both the number and composition of 
housing units they were responsible for had never 
properly reflected the demographics and housing 
demands in their local area, and that this discon- 
nect had worsened over the past decade since the 
province had devolved social-housing responsibil- 
ities to municipalities. We also questioned whether 
the full amount of federal funds provided had 
actually been spent on social housing as required by 
the federal-provincial Social Housing Agreement. 

In its response to our 2009 Annual Report, 
the Ministry indicated that a number of munici- 
pal expenditures were being transferred to the 
provincial level that would free up an estimated 
$1.5 billion annually that could be used to respond 
to social housing and other local priorities. The 



Ministry also committed to working with Service 
Managers to clarify the level of discretion they have 
to change the composition of their social-housing 
units and indicated that it would review its current 
methodology for allocating federal funding. The 
Ministry further committed to considering how best 
to report on how federal funding under the Social 
Housing Agreement was spent. 

At the time of our follow-up, the Ministry 
informed us that it had collaborated on a joint 
working group with its federal and municipal 
counterparts to assess the viability of Canada's 
existing housing stock. The working group's draft 
report, completed in fall 2010, highlighted concerns 
over the viability of the existing stock should the 
federal funding stop and provided a business case 
for the federal government's consideration of 
further investments in social housing. Meanwhile, 
the Ministry's new Strategy commits the province 
of Ontario to engaging other provinces, territories, 
and the federal government in creating a frame- 
work for long-term, flexible funding for affordable 
housing. 

With respect to federal funding allocations, 
the Ministry informed us that it was currently 
reviewing its methodology for distributing these 
monies. This review is scheduled for completion in 
March 2012, at which time Social Housing Agree- 
ment funding for the 2012/13 through 2017/18 
fiscal years is to be finalized and published in the 
Ontario Gazette. 

FUTURE FUNDING INITIATIVES 

Recommendation 5 

To ensure that funding provided achieves the desired 
social-housing impact, the Ministry of Municipal 
Affairs and Housing should require that: 

• each new funding program is supported by a 
detailed business case; and 

• adequate accountability mechanisms for 
reporting on the results achieved by Service 
Managers for the funds provided be put in place 
for all funding programs. 
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In addition, the Ministry should make any neces- 
sary changes to ensure it has the resources and organ- 
izational capacity to properly monitor the effectiveness 
of funding it provides to Service Managers. 

Status 

In our 2009 Annual Report, we found that although 
the province had begun to fund some new housing 
programs, none had been subject to a business-case 
analysis that detailed all the costs and benefits of 
the initiative. We also noted that there were virtu- 
ally no accountability or reporting requirements for 
assessing the impact of the funding provided. 

In its response to our 2009 Annual Report, the 
Ministry indicated that it would review its current 
practice in developing business cases to identify 
and implement any necessary improvements. It also 
committed to reviewing the existing accountability 
mechanisms established for reporting on results by 
municipalities and to assessing its current resource 
requirements to enable it to monitor the effective- 
ness of funding provided to Service Managers. 



At the time of our follow-up, the Ministry 
informed us that it had reviewed the provincial Per- 
formance Measurement Guide and Treasury Board 
submissions related to new funding initiatives, as 
well as program guidelines and agreements. As a 
result of this review, the Ministry advised us that 
sufficient mechanisms were in place that outline 
the accountability and performance measurement 
requirements for these programs. 

The Ministry further informed us that best prac- 
tices had been developed for program guidelines 
and performance measurement that support the 
recognition of the different sizes and capacities 
of the various Service Managers. Based on these 
best practices, guidelines for the rent supplement 
program were being revised. The Ministry also 
informed us that performance measures for the 
initiatives and programs under the new long-term 
Strategy were under development as well. 



Chapter 4 



Ministry of Health and Long-Term Care 



Section 



Teletriage Health 
Services 



Follow-up on VFM Section 3.13, 2009 Annual Report 



Background 



Ontario's teletriage health services provide callers 
from Ontario area codes with free, confidential 
telephone access to a registered nurse for health- 
care advice and information. The services comprise 
Telehealth Ontario — available to all Ontario callers 
24 hours a day, seven days a week — and the Tele- 
phone Health Advisory Service (THAS) — available 
Monday to Friday, 5 p.m. to 9 a.m., and all day 
on weekends and holidays, to 9.5 million patients 
(8.4 million in the 2008/09 fiscal year) enrolled 
with physicians participating in various primary- 
health-care arrangements, such as Family Health 
Teams. For THAS callers, nurses can access the on- 
call physician from the caller's physician's practice, 
and, if needed, the physician may speak directly 
with the caller. 

The Ministry of Health and Long-Term Care 
(Ministry) contracts with a private service pro- 
vider to deliver the teletriage health services. The 
service provider employs almost 300 registered 
nurses at its five call centres located throughout 
Ontario. All calls are handled in one virtual queue, 
with the first available nurse at any of the service 
provider's locations answering the call. The nurses 
use their clinical judgment in conjunction with 
medical decision support software to assist callers. 



During the 2010/11 fiscal year, 896,000 calls were 
responded to by the service provider (905,000 calls 
in 2008/09), and payments to the service provider 
totalled $39 million ($35.1 million in 2008/09). 

We noted in our 2009 Annual Report that the 
Ministry had contracted for the delivery of teletri- 
age health services using a competitive process 
and that the contract included a number of key 
performance requirements, mostly dealing with 
timely access to services. Although only a small 
portion of Ontario's population uses the services, 
our independent survey indicated that those who 
used Telehealth Ontario were generally satisfied. 
However, we believed that improvements could be 
made to both Telehealth and THAS to enhance the 
services and better communicate their availability. 
Our observations included the following: 

• Not only had the number of calls to teletriage 
health services been declining over the previ- 
ous few years, but the number of calls as a 
proportion of the population was significantly 
lower in Ontario than that for similar services 
offered in Alberta and Quebec. Furthermore, 
although over 60% of Ontarians were eligible 
to use THAS, only 1% of eligible individuals 
used it in 2008/09. 

• British Columbia and Quebec had already 
begun using the easily remembered "811" 
phone number for their teletriage health 
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services, and certain other provinces were 
planning to adopt that phone number. Quebec 
reported a 15% increase in call volume fol- 
lowing its implementation. At the time of our 
audit, Ontario had no plans to adopt the "811" 
phone number. 

• The service provider's records indicated that 
about 25% of the callers in the live queue 
hung up before their call was answered. We 
calculated that 85% of callers who waited 
spoke to a nurse within 23 minutes. Eighty- 
five percent of callers who left a call-back 
number spoke to a nurse within 34 minutes. 

• Physicians who were on call to THAS had 
to be paged more than once in over 70% of 
calls requiring a page during 2008, and 9% of 
pages were never returned. 

• Although advice to callers deviated from the 
clinical guidelines and protocols only 5% 

of the time in 2008/09, almost 30% of the 
deviations did not indicate the reason for not 
following the guidelines. 

• Although the proposal submitted by the ser- 
vice provider to the Ministry in 2007 indicated 
that the service provider's nurses would have 
at least three years of any type of nursing 
experience, 20% of our sample of nurses hired 
in 2008 had less experience than that. 

• Because callers were not asked to provide 
their Ontario health card number to the 
service provider, it was not practicable to 
check Ontario Health Insurance Plan records 
to determine whether the caller followed the 
advice given. This would show, for instance, 
whether the teletriage health services were 
influencing callers to use the most appropriate 
health service, such as going to a hospital or 
seeing their family doctor the next day. 

• Unlike most provinces we spoke with, Ontario 
generally did not tape calls for subsequent 
quality assurance review. Rather, the ser- 
vice provider's quality assurance reviewers 
sampled calls only as they were taking place 
and seldom did so during peak periods, when 



nurses experience pressure to respond to wait- 
ing callers within established time frames. 
The quality of advice was also not independ- 
ently evaluated. 

• In 2008/09, the Ministry paid the service pro- 
vider about $39 for each of the first 900,000 
registered calls to teletriage health services 
and about $27 per call after that. Teletri- 
age health services costs for the three other 
provinces that shared cost information with us 
averaged about $20 per call. The Ministry had 
not determined the reason for the significant 
difference. 

• The Ministry had not recently assessed the 
effectiveness of the teletriage health services. 

STANDING COMMITTEE ON 
PUBLIC ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this audit in April 2010. In October 
2010, the Committee tabled a report in the Legis- 
lature resulting from this hearing. The report 
contained nine recommendations and requested 
the Ministry to report back to the Committee with 
respect to the following: 

• the results of its survey on public awareness 
and use of teletriage health services, includ- 
ing measures to ensure that THAS is better 
communicated to patients, and whether new 
measures were being considered to address 
the underutilization of teletriage health servi- 
ces by certain segments of the population, as 
well as the timeline for introducing any new 
measures to address the survey results; 

• the results of the Ministry's business case 
analysis on the costs and benefits of introdu- 
cing an "811" number in Ontario; 

• whether the service provider had changed its 
calculation of the average-wait-time measure 
to start when the call was first received rather 
than starting when the caller was put in the 
queue to wait to talk to a nurse, and any 



Teletriage Health Services 



measures that are being considered to address 
excessive wait times; 

• what steps had been taken to reduce the num- 
ber of unanswered physician pages, including 
the results of discussions with the Ontario 
Medical Association and any technological 
improvements being considered; 

• its assessment, in conjunction with the service 
provider, of the impact of the work-from- 
home option to enhance nurse recruitment 
and retention while maintaining safeguards to 
protect the privacy of callers; 

• the results of the Ministry's deliberations 
on whether callers should be asked for their 
Ontario health card number; 

• whether taping calls to allow for enhanced 
quality assurance processes would be accept- 
able to the Office of the Information and Pri- 
vacy Commissioner, and the Ministry's current 
position on this issue; 

• actions taken by the Ministry to reduce the 
overall cost of the teletriage health services 
program; and 

• the results of the service provider's assess- 
ment of whether an Ontario-based company 
could perform the call centre translation 
services. 

The Ministry formally responded to the Com- 
mittee in March 2011. A number of the issues raised 
by the Committee were similar to our observations. 
Where the Committee's recommendations are 
similar to ours, this follow-up includes the recent 
actions reported by the Ministry to address the 
concerns raised by both the Committee and our 
2009 audit. 



Status of Actions Taken on 
Recommendations 



The service provider as well as the Ministry pro- 
vided us with information in spring 2011 on the 



current status of our recommendations. According 
to this information, significant progress has been 
made in implementing about half of the recommen- 
dations we made in our 2009 Annual Report, while 
some progress has been made on the rest, which 
will require more time to be fully addressed. The 
current status of the actions taken by the service 
provider and the Ministry are summarized follow- 
ing each recommendation. 

ACCESS TO TELETRIAGE SERVICES 

Recommendation 1 

In order to provide more accessible teletriage health 
advice and information, the Ministry should: 

• consider the continued need for a separate 
Telephone Health Advisory Service (THAS) or 
options for increasing the level of awareness 
and acceptance of teletriage services, especially 
among individuab eligible to use THAS and 
among those demographic groups, such as sen- 
iors, that underutilize the services; and 

• explore the use of an easily remembered phone 
number, such as "811 " (which is used or being 
planned for in several other large provinces), for 
both Telehealth Ontario and THAS. 

Status 

At the time of our follow-up, the Ministry indicated 
that it has a contractual obligation with physicians 
to provide THAS and that any changes would 
be discussed in upcoming negotiations with the 
Ontario Medical Association. The Ministry also 
noted that it completed a survey in March 2010 that 
provided the Ministry with information on the level 
of Ontarians' awareness of teletriage health servi- 
ces. Commencing in November 2010, and partly in 
response to the survey information, the Ministry 
conducted a campaign meant to educate Ontarians 
regarding health-care choices, including Telehealth 
Ontario. The Ministry further indicated that sig- 
nificantly more calls were received by the service 
provider during December 2010 and January 2011, 
including significantly more calls to THAS. The 
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Ministry also noted that although the number of 
calls increased, the caller demographic groups 
remained similar to other months. 

As part of a 2010 jurisdictional scan of teletriage 
health services in five other Canadian jurisdictions, 
the Ministry included questions to help it explore 
the use of an easily remembered phone number, 
such as "811." The Ministry found that provinces 
that implemented an "811" number subsequently 
experienced increases of up to 15% in the volume of 
calls made to their teletriage health services. At the 
time of our follow-up, the Ministry indicated that 
an "811" number could be implemented in Ontario 
only if any increased call volumes could be handled 
by the service provider with little to no increase in 
total funding. In this regard, the Ministry noted that 
the service provider has implemented a couple of 
initiatives to handle more calls without increasing 
the resources used to do so. As yet, however, no 
decision has been made on whether or not to intro- 
duce "811" in Ontario. 

CALL MANAGEMENT 

Recommendation 2 

To help ensure that all callers' questions are answered 
within a reasonable time frame, the Ministry should: 

• ask the service provider to instruct its nurses 
to redirect information requests for phone 
numbers and addresses of community services to 
non-nursing staff; 

• review alternative ways to promote timely phys- 
ician responses to pages for Telephone Health 
Advisory Service callers, such as financial penal- 
ties when on-call physicians do not respond 
when paged or financial incentives for those 
physicians who consistently exceed standards; 
and 

• require the service provider to measure the 
wait time for callers from the time the call was 
initially received for both the live and call-back 
queues. 

As well, to ensure that caller information remains 
confidential: 



• the service provider should sign agreements 
with its vendors that handle confidential caller 
information, such as those providing its transla- 
tion and off-site storage services, to maintain 
appropriate physical and electronic security, in 
accordance with its contract with the Ministry; 
and 

• the Ministry should ensure that periodic vulner- 
ability and penetration testing is completed at 
the service provider to identify and correct any 
security weaknesses. 

Status 

The Ministry indicated that, effective March 2011, 
the role of the service provider's non-clinical staff 
was expanded to answer calls requesting informa- 
tion about community services, rather than having 
nurses answer these calls. The service provider 
further clarified that although the non-clinical 
staff now respond to callers requesting information 
on local services, nurses still respond to callers 
requesting health information or requiring symp- 
tom assessment. 

The Ministry sent two bulletins to physicians 
in fall 2010 regarding after-hours requirements, 
which highlighted the physician's responsibilities 
for responding to pages from the Telephone Health 
Advisory Service. Further, the Ministry indicated 
that it now receives and reviews monthly reporting 
from the service provider regarding how many 
pages, for each group of primary-care physicians, 
are not being answered within 30 minutes, and how 
many are not answered at all. However, at the time 
of our follow-up, the Ministry noted that financial 
penalties could not be imposed for pages not 
returned within 30 minutes, because its agreements 
with the physicians do not include time limits for 
answering pages. Potential changes to the existing 
agreements would have to be negotiated with the 
Ontario Medical Association. 

At the time of our follow-up, the Ministry indi- 
cated that, because of technology limitations, the 
service provider was unable to measure wait times 
for callers from the time a call is initially received. 
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However, the Ministry now receives reports on and 
monitors the wait times experienced by callers who 
remain on the phone in the live queue, as well as 
wait times for callers to receive a call back from a 
nurse. 

With respect to maintaining appropriate 
physical and electronic security for confidential 
caller information, the service provider indicated 
that it signed agreements with its vendors in early 
2010 that included requirements related to the Per- 
sonal Health Information Protection Act, in accord- 
ance with its contract with the Ministry 

The Ministry noted that a Threat Risk Assess- 
ment for teletriage health services was completed in 
2008, and that penetration testing was completed 
in March 2011. The Ministry, in conjunction with 
the service provider, reviewed the results of the 
penetration testing and determined that, while 
there were no urgent security issues, the items 
noted would be followed up. 

ADVICE TO CALLERS 

Recommendation 3 

To better ensure that callers to teletriage services 
receive and follow the most appropriate advice to 
address their health concerns, the service provider 
should: 

• hire nurses who have at least three years of 
nursing experience, including at least one year 
of acute-care or clinical experience, in accord- 
ance with its proposal to secure the contract 

to provide teletriage services and its internal 
policies; 

• ensure that nurses complete their ongoing train- 
ing in accordance with policies; and 

• require nurses to document the reason for 
providing advice that does not follow a clinical 
guideline or protocol. 

As well, to better determine the impact of the 
advice provided to callers, the Ministry, in conjunc- 
tion with the service provider, should develop a pro- 
cess (such as obtaining Ontario health card numbers 
and following up on a sample of the callers' subse- 



quent actions) for periodically assessing the extent to 
which callers follow the nurses' advice. 

Status 

At the time of our follow-up, the Ministry indicated 
that it had discussed the qualifications of newly 
hired nurses with the service provider, and as a 
result, the service provider had committed to hiring 
only nurses with at least three years of experience, 
including at least one year of acute-care or clinical 
experience. The service provider noted that all 
nurses hired since March 2010 had these qualifica- 
tions at the time of hire. 

The Ministry noted that the service provider 
has established a new quality assurance depart- 
ment, which has implemented nurse training and 
coaching schedules. The service provider further 
commented that the revised ongoing training 
requirements for nurses commenced January 1, 
2010, and that attendance is tracked. Information 
on compliance with the training requirements 
is reported on a monthly basis to the service 
provider's quality assurance department and on a 
quarterly basis to the Medical Liaison Committee, 
of which the Ministry is a member. 

The Ministry stated that the service provider 
has adjusted its call management process such that 
nurses are now required to document, before com- 
pleting a call, the reason for providing advice that 
deviates from clinical protocols. 

At the time of our follow-up, the Ministry 
indicated that it was continuing to review the feas- 
ibility of obtaining Ontario health card numbers 
for the purpose of tracking whether callers fol- 
lowed the advice provided by the teletriage health 
service's nurses. The Ministry expected to have a 
decision by fall 2011. The Information and Privacy 
Commissioner advised the Ministry that obtaining 
Ontario health card numbers is acceptable from a 
privacy perspective. However, the Ministry indi- 
cated that it remained concerned that collecting 
Ontario health card numbers could cause wait 
times for callers as well as costs to increase because 
the average length of calls could increase due to 
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the time it takes callers to find their health card. 
As an alternative, the Ministry is considering a 
project to determine the extent to which callers to 
the Telephone Health Advisory Service (THAS) fol- 
low the advice they receive, because calls to THAS 
are automatically matched to Ontario health card 
numbers through the callers' physicians. The Min- 
istry also indicated that it would be conducting an 
external evaluation of the teletriage health services 
to address whether the services provide appropri- 
ate health advice to Ontarians and are useful. This 
evaluation was expected to commence in fall 2011 
and be completed by summer 2012. 

QUALITY ASSURANCE 

Recommendation 4 

To better ensure the quality of teletriage services and 
identify areas for improvement: 

• the service provider should have independent 
reviewers conduct an established number of ran- 
dom audits on calls received at different times 
of the day and on different days of the month, 
including weekends and holidays; 

• the service provider should periodically analyze 
the overall issues noted in call audits and com- 
plaints by call centre and by nurse to determine 
whether there are any systemic issues or trends 
that warrant follow-up; and 

• the Ministry should conduct periodic independ- 
ent satisfaction surveys of individuals impacted 
by teletriage services, including callers, phys- 
icians, and emergency department staff. 

The Ministry should request the Information and 
Privacy Commissioner's input on whether calls to the 
service provider can be taped for periodic review to 
determine the appropriateness of advice provided by 
teletriage nurses. If calls are not taped for periodic 
review, the Ministry should seek another way to 
obtain independent assurance on the appropriateness 
of advice provided by teletriage nurses (for example, 
through the use of mystery callers). 



Status 

At the time of our follow-up, the service provider 
indicated that it established its new quality assur- 
ance department in August 2010 and implemented 
updated quality monitoring processes in Septem- 
ber 2010. Quality analysts, who do not have a direct 
reporting relationship with the nurses they review, 
monitor a random sample of calls each month. 
These samples comprise calls received at various 
times of the day and on various days of the month, 
including weekends and holidays. The service 
provider's revised policy requires that each quality 
analyst review at least eight calls a day, so that a 
minimum of two calls per nurse are monitored each 
month. 

The service provider stated that issues identi- 
fied as a result of call audits and complaints are 
reviewed at each call centre monthly. With respect 
to call audits, the service provider indicated that it 
was developing a process for analyzing trends by 
call centre and nurse, which it planned to imple- 
ment by fall 2011. With respect to complaints, the 
service provider noted that it tracks data by nature 
of complaint, because detailed data are not readily 
available that would allow analysis of complaints 
by call centre or by nurse. The service provider uses 
this information to identify systemic complaints. 
The service provider noted that in 2009, it intro- 
duced additional training initiatives to address 
systemic issues that had been identified. 

The Ministry noted that the previously men- 
tioned evaluation of the teletriage health services 
will include satisfaction surveys for teletriage 
stakeholders, including callers, physicians, and 
emergency department staff. 

The Ministry stated that the Information and 
Privacy Commissioner had advised that recording 
calls for the purpose of quality assurance would 
be acceptable as long as the callers were told in 
advance and could request not to be recorded. As a 
result, the service provider is now recording calls, 
with selected calls being reviewed by its quality 
assurance team to determine whether appropriate 
advice is being provided by teletriage nurses. As 
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well, the Ministry indicated that it is arranging an 
internal audit, anticipated to start in winter 2012, 
which will review among other things the service 
provider's adherence to its standards and processes 
for call recording and quality assurance. 

PAYMENTS FOR TELETRIAGE SERVICES 

Recommendation 5 

To ensure that the amount paid for teletriage services 
is reasonable in comparison to other jurisdictions and 
in accordance with the Ministry's contract with the 
service provider, the Ministry should: 

• obtain information on the delivery of teletriage 
services in other provinces to determine whether 
there are areas where Ontario's teletriage servi- 
ces could be delivered more economically; and 

• confirm that payments made to the Ontario 
Pharmacists' Association's Medication Informa- 
tion Service are reasonable, based on the actual 
number of calls that the Telehealth Ontario 
service provider reports having referred to the 
Medication Information Service. 

Status 

The Ministry's previously mentioned 2010 jurisdic- 
tional scan of teletriage health services included 
a request for information regarding the cost per 
call of the services. According to the Ministry, it 
received only high-level information on costs in 
other Canadian jurisdictions, and these costs varied 
based on the different standards and types of servi- 
ces offered in these jurisdictions. Therefore, to help 
ensure that Ontario's teletriage health services are 
as cost-effective as possible, the Ministry obtained 
a proposal from the service provider to modify 
the current call management process with a goal 
of creating efficiencies. According to the Ministry, 
one resulting change, effective March 2011, was 
to expand the role of the service provider's non- 
clinical staff to include answering calls requesting 
information about community services. This change 
enables nurses, who previously handled these 
information requests, to be available for additional 



health-care-related calls. Further, the Ministry 
noted that in conjunction with the service provider, 
and with input from the College of Nurses of 
Ontario (which governs both registered nurses and 
registered practical nurses), it is exploring the pos- 
sibility of having registered practical nurses (RPNs), 
rather than registered nurses, answer certain types 
of calls to the teletriage health services. The service 
provider indicated that the use of non-clinical staff, 
as well as the possible use of RPNs, where appro- 
priate, could reduce the cost per call of teletriage 
health services in Ontario. The cost per call has 
risen to more than $43, about a 12% increase since 
the 2008/09 fiscal year. As well, the Ministry noted 
that its planned evaluation of the teletriage health 
services may also help in determining ways in 
which calls could be managed more efficiently. 

With respect to ensuring that payments made 
to the Ontario Pharmacists' Association's Medica- 
tion Information Service are based on the actual 
number of calls it handles, the Ministry stated that 
the Association and the service provider now meet 
monthly to reconcile call volumes. The Ministry 
indicated that this approach has resolved the con- 
cern we raised during our 2009 audit. 

EFFECTIVENESS OF TELETRIAGE 
SERVICES 

Recommendation 6 

To better ensure that teletriage services are meeting 
their objectives, the Ministry, in conjunction with 
the service provider, should expand the performance 
standards to include indicators on callers who wait in 
the live queue (including how long they wait and how 
many hang up before speaking to a nurse) and on the 
quality of the nurses' advice. 

As well, because it has been almost five years since 
the effectiveness of the teletriage services in meeting 
their established objectives has been assessed, the 
Ministry should consider conducting a formal evalua- 
tion. One area to consider including in the evaluation 
is an assessment of whether using a teletriage service 
improves callers' health-related decision-making. 
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At the time of our follow-up, the Ministry indicated 
that in April 2010 it began receiving monthly 
reports from the service provider on wait times 
for callers in the live queue. More specifically 
these reports indicate how long callers wait in the 
live queue until one of the following three events 
occurs: a nurse answers the call; the caller leaves 
a message asking to be called back; or the caller 
hangs up. The Ministry also indicated that in 
April 2011 it began receiving monthly reports from 
the service provider on the results of its call audits, 
which reflect the quality of the nurses' advice to 
callers. 

The Ministry noted that the previously men- 
tioned evaluation of the teletriage health services 
will review whether the services are meeting their 
intended objectives, including whether the pro- 
gram is improving consumer health education and 
callers' health-related decision-making. It will also 
identify ways to better meet those objectives. 
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Section 



I 4.14 Unfunded Liability of the 
Workplace Safety and 



Insurance Board 



Follow-up on VFM Section 3.14, 2009 Annual Report 



Background 




The Workplace Safety and Insurance Board (WSIB) 
is a statutory corporation created by the Workplace 
Safety and Insurance Act, 1997 (Act). Its primary 
purpose is to provide income support and fund 
medical assistance to workers injured on the job. 
The WSIB receives no funding from the govern- 
ment; it is financed through premiums charged on 
the insurable payrolls of employers. The govern- 
ment has the sole responsibility for setting benefits 
and coverage through legislation, while the WSIB 
has responsibility for setting premium rates. 

In our 2005 Annual Report, we noted that the 
assets in the WSIB's insurance fund were substan- 
tially less than what was needed to satisfy the 
estimated lifetime costs of all claims currently in 
the system, thus producing what is known as an 
"unfunded liability," which stood at $6.4 billion at 
that time. 

In the review that appeared in our 2009 Annual 
Report, we observed that, as of December 31, 2008, 
the unfunded liability stood at $11.5 billion, an 
increase of $3.4 billion from the previous calendar 
year (by December 31, 2010, the unfunded liability 



was $12.4 billion and had almost doubled in size 
since 2006) . Our review expressed the concern that 
the growth and magnitude of the unfunded liability 
posed a risk to the system's financial viability and 
ultimately could result in the WSIB being unable 
to meet its existing and future financial com- 
mitments to provide worker benefits. Eliminating or 
reducing the unfunded liability required that four 
key levers — legislated benefits, coverage, premium 
rates, and investments — work effectively in tandem. 
We observed that the WSIB and the government 
may have to commit to a different strategy with 
respect to these levers if the unfunded liability is to 
be addressed within a reasonable period of time. 
Our other observations included the following: 

• The WSIB's funding ratio of assets to liabilities 
was 53.5%, considerably lower than that of 
any of the four other large provincial boards 
we reviewed in British Columbia, Alberta, 
Manitoba, and Quebec, which averaged 
102%. In each of these four provinces, legisla- 
tive and policy differences are key factors that 
contributed to their higher funding ratios. 

• The WSIB and governments have sought over 
the last two decades to satisfy simultaneously 
two major stakeholders: employers, who 
wanted lower premiums, and workers, who 
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wanted higher benefits. This has undoubtedly 
affected the size of the current unfunded 
liability. 

• The WSIB's ability to eliminate the unfunded 
liability has to some extent been limited by 
the government's control over benefit changes 
and over which businesses and industries 

are covered by the system. For example, in 
Ontario, 72.6% of the workforce was covered 
by the system as of 2007, compared to 93.1% 
in British Columbia and 93.4% in Quebec. 

• Annual premium revenues in recent years 
have not been enough to cover benefit costs. 
Premiums have increased by an average of 
only 1% each year since 2001, at the same 
time as the WSIB was reporting average 
annual deficits of more than $900 million. 

• Benefit and health-care costs have risen stead- 
ily over the last 10 years as a result of workers 
staying on benefits longer and receiving 
increases in those benefits as a result of legis- 
lative changes. 

• The WSIB's 15-year average rate of return on 
its investments from 1994 to 2008 was 6.6%. 
Given that future benefit costs are expected 
to rise at 7% annually, investments must earn 
more than 7% before any reduction of the 
unfunded liability can be realized solely from 
investment returns. 

Our 2009 review of the unfunded liability did 
not make specific recommendations, but rather 
discussed the factors contributing to the growth 
of the unfunded liability and the initiatives being 
undertaken by the WSIB to address it. The WSIB 
responded to the issues we raised and acknow- 
ledged that it would need to take significant 
actions to get its financial affairs in order. We have 
structured this follow-up of our review on the basis 
of discussions with the President and other senior 
officials of the WSIB, and a formal written update 
the WSIB provided to us. 



STANDING COMMITTEE ON PUBLIC 
ACCOUNTS 

The Standing Committee on Public Accounts held 
a hearing on this review in February 2010. In 
October 2010, the Committee tabled a report in the 
Legislature resulting from this hearing. The report 
contained 10 recommendations and requested that 
either the Ministry of Labour or the WSIB report 
back to the Committee with respect to the following: 

• whether the Ministry believes that the WSIB 
should continue to govern its own financial 
affairs and address the unfunded liability 
itself, including the Ministry's views of the 
benefits and drawbacks of opening up WSIB 
appointments to public application; 

• the outcome of the WSIB's consultations on 
whether there is support for legislative chan- 
ges that would require the WSIB to become 
fully funded in time; 

• the outcome of the WSIB's review of premium 
rate-setting, including a timeline and the 
expected impact on premium rates if the 
review recommends changes to the way they 
are set; 

• the WSIB's strategy to manage rising occupa- 
tional disease claims and the impact it antici- 
pates these claims will have on its unfunded 
liability; 

• the outcomes of the Ministry's examination of 
its options for more comprehensive coverage 
levels for Ontario workers; 

• the WSIB's assessment of how it expects 
implementation of changes to the Labour 
Market Re-entry (LMR) Program to impact 
both the duration of claims and the unfunded 
liability; 

• the outcome of the WSIB's implementation of 
its narcotic control program, including cost 
savings that have accrued from it and whether 
it has had an impact on the duration of claims; 

• the status of the implementation of recom- 
mendations made in the Chair's Report on 
Stakeholder Consultations; 
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• whether the WSIB had achieved its target of a 
7% reduction in new claims in 2009 and, if not, 
the action it had taken in 2010 on this issue; and 

• what progress the WSIB had made in drafting 
a strategy by December 31, 2010, to address 
its unfunded liability, including the results 

of its anticipated strategic plan and planned 
reduction in the unfunded liability over the 
next five years. 
Formal responses to the Committee's recom- 
mendations were provided by the WSIB on Decem- 
ber 14, 2010, and April 4, 2011, and by the Ministry 
of Labour (Ministry) on February 2, 2011. Where 
the Committee's recommendations were similar 
to ours, this follow-up includes the recent actions 
reported by the Ministry and the WSIB to address 
the observations raised by both the Committee and 
our 2009 audit. 







btatus ot Actions la 
Issues Raised 
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According to the information we received from the 
WSIB and discussions with senior management 
regarding the issues raised in our 2009 review, the 
WSIB had made progress in introducing a number 
of initiatives to address the unfunded liability. As 
well, legislation has been passed that, subject to 
proclamation, would require that the WSIB reach 
a prescribed level of funding within a specified 
time frame. The funding and time frame are to be 
established by regulation that will take the results 
of the current independent funding review into 
consideration. 

The following update has been organized on 
the basis of the key initiative areas identified in the 
WSIB's 2011-2013 Corporate Business Plan. We 
have prefaced each section with relevant observa- 
tions from the review that appeared in our 2009 
Annual Report. 



SUFFICIENT FUNDING— FOCUS ON THE 
UNFUNDED LIABILITY AND STRIVE TO 
SIGNIFICANTLY REDUCE IT 

Observations 

• Both the WSIB and the government may have 
to commit to a different strategy with respect 
to the setting of premium rates and benefits if 
the WSIB is to eliminate the unfunded liability 
within a reasonable period. 

• Section 96 (2) of the Act states: "The Board 
has a duty to maintain the insurance fund so 
as not to unduly or unfairly burden any claims 
of Schedule 1 employers in futureyears with 
payments under the insurance plan in respect 
of accidents of previous years." Clearly, the very 
existence of the unfunded liability demonstrates 
that, over the years, the province's employers 
have not fully funded the costs of injuries and 
occupational diseases, so these liabilities will 
need to be funded by future employers. Thus, 
employers in currently declining industry sec- 
tors have transferred workplace-safety financial 
obligations to other current and future genera- 
tions of employers. 

• Eliminating or reducing the unfunded liability 
requires the interaction of four key levers — legis- 
lated benefits, coverage, premium rates, and 
investments — to work effectively in tandem. The 
inability to eliminate the WSIB's unfunded lia- 
bility over the last two decades has been owing 
in part to the WSIB's desire to satisfy all the 
stakeholders. 

Status 

The WSIB indicated that the following actions had 
been taken in response to these issues: 

• The WSIB launched an independent funding 
review seeking advice from stakeholders. The 
review, led by an external academic, provides 
the opportunity for employers, workers, and 
other interested parties to make presenta- 
tions. It is designed to provide the WSIB with 
advice on issues such as how to achieve full 
funding of the insurance fund, the design of 
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the employer incentive programs, and the 
effectiveness of the rate-group structure and 
the premium-setting methodology. 

• Legislative amendments to the Act have been 
passed and proclaimed by the Legislature. 
These amendments provide the WSIB with 
more autonomy to govern its own financial 
affairs. 

• Legislative amendments to both the Workplace 
Safety and Insurance Act and the Occupational 
Safety and Insurance Act (Bill 160) have been 
passed with the intent of promoting the 
integration of the prevention and enforce- 
ment elements of the occupational health and 
safety system. These measures include the 
transfer of the WSIB's prevention mandate to 
the Ministry of Labour. This will also allow the 
WSIB to concentrate on its insurance function. 

• The WSIB established an Actuarial Advisory 
Committee to provide general advice and 
counsel to the President and CEO. 

REVENUE MUST COVER COSTS- 
OPTIMIZE PREMIUM AND INVESTMENT 
REVENUES AS A CRITICAL MEASURE OF 
FISCAL HEALTH 

Observations 

• Premium revenues have not increased enough to 
offset the costs of the benefits that are mandated 
under the Act. Benefit expenses increased by 
about 7% annually from 1999 through 2008, 
but premium revenues increased by an average 
of only 3% during the same period. 

• Ontario will eventually need to increase its 
premium rates if it hopes to make any progress 
toward eliminating its unfunded liability — 
unless downward revisions are made to the 
current benefits structure or investment returns 
recover dramatically. 

• Having too few investments relative to the 
WSIB's liabilities and liquidating investments 
to pay current operating expenses and benefit 
claims typically have a significant negative 



impact on the size of the unfunded liability and 
fiscal sustainability of the WSIB. 

Status 

The WSIB indicated that, in response to these 
issues, it had: 

• put into place a 2% increase in the average 
annual premium rate for 2011, with a further 
2% increase planned for 2012; 

• begun addressing the sources of revenue leak- 
age, including employers' arrears and pay- 
ment avoidance, as well as non-compliance 
strategies; and 

• implemented its Strategic Investment Plan 
reflecting a more conservative investment 
strategy with a focus on reduced volatility. 

RIGHT-SIZING COSTS-REDUCE TOTAL 
BENEFIT COSTS THROUGH REDUCING 
WORKPLACE FATALITIES, INJURIES, AND 
ILLNESSES, AND PROMOTING EARLY 
RECOVERY AND RETURN TO WORK 

Observations 

• Benefit and health-care costs have been rising 
over the last 10 years. These cost increases — in 
particular, benefit cost increases arising from 
increases in the amount of time that workers 
are staying on benefits and increases in benefits 
arising from legislative changes — have contrib- 
uted to the unfunded liability. 

• Health-care costs paid by the WSIB on behalf 
of workers receiving benefits averaged 16% of 
total benefit costs over the 1999-2008 period. 
But in that same period, these health-care costs 
more than doubled — rising from $238 million 
in 1998 to $619 million in 2008. One of the 
primary drivers of increased health-care costs is 
the increased number of narcotic prescriptions 
for analgesia (pain relief). 

• Employer incentive programs were not providing 
the desired outcomes. If claims duration in gen- 
eral is increasing, rebates should decrease and/ 
or surcharges should increase correspondingly. 
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A study noted that the opposite was occurring: 
employers were still being rewarded even as 
their injured-worker claims duration was 
increasing. 

Status 

The WSIB indicated that it had taken the following 
actions in response to these issues: 

• It had implemented a new Work Reintegration 
Model to improve return- to-work outcomes. 
The model involves more early involvement 
and the use of work transition specialists. 

• It was more carefully managing employer 
incentive programs, with the result that the 
imbalance of refunds exceeding surcharges 
was the lowest that it had been in the past 16 
years. 

• It had introduced a more appropriate gradu- 
ated narcotic therapy for injured workers. 

• It had initiated a value-for-money audit to 
report on the effectiveness and efficiency of 
the WSIB's claims management process. 
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Review of Government 
Advertising 



INTRODUCTION 

In reviewing our activities this past year with 
regard to the Government Advertising Act, 2004 
(Act), I wanted first to highlight an observation 
made in Australia, where the national and state 
governments have considered a variety of actions 
to ensure that public funds do not pay for partisan 
ads. Specifically, I was heartened to come across 
this recommendation in the 2008 report of a Legis- 
lative Committee in New South Wales, Australia: 
"That the Premier entrust the Auditor General with 
oversight responsibility for government advertising, 
with the Auditor General's powers to be modelled 
on those of the Auditor General in Ontario, Can- 
ada [emphasis added]." 

It is encouraging to note that after six years of 
existence, the Act may well be the gold standard by 
which other jurisdictions measure themselves in 
their drive to ensure that no public money is spent 
on partisan advertising. 

The Act took effect in December 2005 after two 
years of debate in the Legislature — and several 
years of discussion prior to that — as legislators 
questioned the appropriateness of a government 
using public funds for advertising that could be 
considered to further its own partisan interests. 

The main intent of the Act is to prohibit govern- 
ment advertising that may be viewed as promoting 
the governing party's political interests by fostering 
a positive impression of the government or a nega- 
tive impression of any group or person critical of 



the government. Under the Act, most government 
advertisements must be submitted to and approved 
by the Auditor General before they can be used. The 
full text of the Act can be found at www.e-laws. 
gov.on.ca. 

This chapter, which satisfies the legislative 
requirements in the Act as well as in the Auditor 
General Act to report annually to the Legislative 
Assembly, outlines the work we have done over the 
past year to ensure that the Act is adhered to. 



Overview of the Advertising 
Review Function 



Under the Act, the Auditor General is responsible 
for reviewing specified types of government adver- 
tisements to ensure that they meet legislated stan- 
dards. Above all, they must not contain anything 
that is, or could be interpreted as being, primarily 
partisan in nature. 

The Act outlines standards each advertisement 
must meet and states that "an item is partisan if, 
in the opinion of the Auditor General, a primary 
objective of the item is to promote the partisan pol- 
itical interests of the governing party." 

The Act also provides the Auditor General with 
the discretionary authority to consider additional 
factors in determining whether a primary object- 
ive of an item is to promote the partisan political 
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interests of the governing party (see the "Other 
Factors" section later in this chapter). 

WHAT FALLS UNDER THE ACT 

The Act applies to advertisements that govern- 
ment offices — specifically, government ministries, 
Cabinet Office, and the Office of the Premier — pro- 
pose to pay to have published in a newspaper or 
magazine, displayed on a billboard, or broadcast on 
radio or television. It also applies to printed matter 
that a government office proposes to pay to have 
distributed to households in Ontario either by bulk 
unaddressed mail or by another method of bulk 
delivery. Advertisements meeting any of these def- 
initions are known as "reviewable" items and must 
be submitted to my Office for review and approval 
before they can run. 

The Act excludes from review any job advertise- 
ment or notice to the public required by law. Also 
excluded are advertisements on the provision of 
goods and services to a government office and those 
on urgent matters affecting public health or safety. 

Although the following are not specifically 
excluded by the Act, we have come to a mutual 
understanding with the government that they are 
not subject to the Act: 

• electronic advertising on government web- 
sites or any public site, except for web pages 
identified and promoted in a reviewable item 
(see the "Websites" subsection later in this 
chapter) ; and 

• brochures, pamphlets, newsletters, news 
releases, consultation documents, reports, 
and other similar printed matter, materials, or 
publications. 

The Act requires government offices to submit 
every reviewable item to the Auditor General's 
Office for review. The government office cannot 
publish, display, broadcast, distribute, or dissemin- 
ate the submitted item until the head of that office, 
usually the deputy minister, receives notice, or is 
deemed to have received notice, that the advertise- 
ment has been approved. 



The Auditor General's Office, by regulation, has 
seven business days to render its decision. If we do 
not give notice within this time, the government 
office is deemed to have received notice that the 
item meets the standards of the Act, and the item 
may be run. 

If my Office notifies the government office that 
the item does not meet the Act's standards, the item 
may not be used. However, the government office 
may submit a revised version of the rejected item 
for another review. As with the first submission, my 
Office has seven days to render its decision. 

Once an item has been approved, a government 
office may use it for the next 12 months. However, 
my Office can rescind an approval if we determine 
that new circumstances have changed the context 
in which the ad appears. Under the Act, all deci- 
sions of the Auditor General are final. 

A pre-review is also available to government 
offices wishing us to examine an early version of an 
item. This can be a script or storyboard, provided 
that it reasonably reflects the item as it is intended 
to appear when completed. Pre-reviews help limit 
the investment of time and money spent to develop 
items containing material that could be deemed 
objectionable under the Act. 

If material submitted for pre-review appears 
to violate the Act, we provide an explanation to 
the government office. If it appears to meet the 
standards of the Act, we so advise the government 
office. However, before the item can be used, the 
government office must submit the finished item for 
review to ensure that it still meets the standards of 
the Act. 

A pre-review is strictly voluntary on our part and 
is outside the statutory requirements of the Act. 

STANDARDS FOR PROPOSED 
ADVERTISEMENTS 

In conducting its review, the Auditor General's 
Office first determines whether the proposed adver- 
tisement meets the standards of the Act. These are: 
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• The item must be a reasonable means of 
achieving one or more of the following 
objectives: 

• to inform the public of current or proposed 
government policies, programs, or services; 

• to inform the public of its rights and 
responsibilities under the law; 

• to encourage or discourage specific social 
behaviour in the public interest; and/or 

to promote Ontario, or any part of the prov- 
ince, as a good place to live, work, invest, 
study, or visit, or to promote any economic 
activity or sector of Ontario's economy. 

• The item must include a statement that it is 
paid for by the government of Ontario. 

• The item must not include the name, voice, or 
image of a member of the Executive Council 
(cabinet) or a member of the Legislative 
Assembly (unless the primary target audience 
is located outside Ontario, in which case the 
item is exempt from this requirement) . 

• The item must not have as a primary objective 
the fostering of a positive impression of the 
governing party, or a negative impression of a 
person or entity critical of the government. 

• The item must not be partisan; that is, in the 
opinion of the Auditor General, it cannot have 
as a primary objective the promotion of the 
partisan interests of the governing party. 

OTHER FACTORS 

In addition to the specific statutory standards 
above, the Act allows the Auditor General to 
consider additional factors to determine whether 
a primary objective of an item is to promote the 
partisan interests of the governing party. In gen- 
eral, these additional factors relate to the overall 
impression conveyed by the ad and how it is likely 
to be perceived. Consideration is given to whether 
it includes certain desirable attributes and avoids 
certain undesirable ones. These are: 

• Each item should: 



• contain subject matter relevant to govern- 
ment responsibilities (that is, the govern- 
ment should have direct and substantial 
responsibilities for the specific matters 
dealt with in the item) ; 

• present information objectively, in tone and 
content, with facts expressed clearly and 
accurately, using unbiased and objective 
language; 

• emphasize facts and/or explanations, not 
the political merits of proposals; and 

• enable the audience to distinguish between 
fact on the one hand and comment, opin- 
ion, or analysis on the other. 

• Items should not: 

• use colours, logos, and/or slogans com- 
monly associated with any recognized 
political party in the Legislative Assembly 
of Ontario; 

• inappropriately personalize (for instance, 
by attacking opponents or critics) ; 

• directly or indirectly attack, ridicule, or 
criticize the views, policies, or actions of 
those critical of the government; 

• be aimed primarily at rebutting the argu- 
ments of others; 

• intentionally promote, or be perceived as 
promoting, political-party interests (to 
this end, consideration is also given to 
such matters as timing of the message, the 
audience it is aimed at, and the overall 
environment in which the message will be 
communicated) ; 

• deliver self-congratulatory or political- 
party image-building messages; 

• deal with matters such as a policy proposal 
where no decision has yet been made, 
unless the item provides a balanced 
explanation of both the benefits and the 
disadvantages; 

• present pre-existing policies, products, ser- 
vices, or activities as if they were new; or 

• use a uniform resource locator (URL) to 
direct readers, viewers, or listeners to a 



"first click" web page with content that may 
not meet the standards required by the Act 
(see "Websites" in the following section) . 

OTHER REVIEW PROTOCOLS 

Since taking on responsibility for reviewing govern- 
ment advertising, my Office has tried to clarify, in 
co-operation with government offices, areas where 
the Act is silent. What follows is a brief discussion 
of the main areas that have required clarification 
over the years. 

Websites 

Although websites are not specifically reviewable 
under the Act, we believe that a website used in 
an advertisement is seen as an extension of the 
ad. Following discussions with the government, 
we came to an agreement that the first page or 
"click" of a website accessed by using the URL in a 
reviewable item would be included in our review. 
We agreed not to consider web pages beyond the 
first click, unless that first click is a gateway page, 
in which case we review the next page. We examine 
reviewable web pages for any information or mes- 
sages that may not meet the standards of the Act. 
For example, the page must not include a minister's 
name, voice, or photograph, nor deliver self- 
congratulatory, party image-building messages, 
or messages that attack the policies, opinions, or 
actions of others. 

Event/Conference Program Advertisements 
and Payments in Kind 

Government advertisements sometimes appear 
in programs and other materials distributed at 
public events such as conferences, trade shows, and 
exhibitions. In considering this type of advertise- 
ment, we concluded that it should be subject to the 
Act because the programs usually follow the same 
format and serve a similar purpose as magazines 
and other print media. On the issue of payment for 
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these advertisements, government offices often 
make in-kind or financial contributions to an event, 
including paid sponsorship. Therefore, we consider 
the "free" advertisement to have been indirectly 
paid for. 

Our rationale was based on the fact that the free 
advertisement is typically granted after the govern- 
ment office has made a financial contribution or 
sponsored the event. Government officials have 
agreed with this approach, and these items must be 
submitted for review. 

Third-party Advertising 

Government funds provided to third parties are 
sometimes used for advertising. The government 
and my Office have agreed that third-party adver- 
tising must be submitted for review if it meets all of 
the three following criteria: 

• a government office provides the third party 
with funds intended to pay part or all of the 
cost of publishing, displaying, broadcasting, 
or distributing the item; 

• the government grants the third party permis- 
sion to use the Ontario logo or another official 
provincial visual identifier in the item; and 

• the government office approves the content of 
the item. 

Government Recruitment Advertisements 

As previously noted, the Act excludes job adver- 
tisements from review. We have interpreted this 
exemption as applying to advertising for specific 
government jobs, but not to broad-ranging generic 
recruitment campaigns. The government has 
agreed with our interpretation and, as a result, 
generic recruitment campaigns must be submitted 
to my Office for review. 
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External Advisers 



Under the Auditor General Act, the Auditor General 
can appoint an Advertising Commissioner to assist in 
fulfilling the requirements of the Government Adver- 
tising Act, 2004. However, instead of appointing one 
Advertising Commissioner, my Office has engaged 
a number of external advisers to assist us in the 
ongoing review of items submitted for review. The 
following advisers have been engaged by my Office 
during the 2010/11 fiscal year: 

« Rafe Engle is a Toronto lawyer specializing in 
advertising, marketing, communications, and 
entertainment law. He is also the outside legal 
counsel for Advertising Standards Canada, 
and Chair of its National Consumer Response 
Council. Before studying law, Mr. Engle 
acquired a comprehensive background in 
media and communications while working in 
the advertising industry. 

• Jonathan Rose is Associate Professor of 
Political Studies at Queen's University. He is 
a leading Canadian academic with interests 
in political advertising and Canadian politics. 
Professor Rose has written a book on govern- 
ment advertising in Canada and a number of 
articles on the way in which political parties 
and governments use advertising. 

• Joel Ruimy is a Toronto communications 
consultant with three decades of experience 
as a journalist, editor, and producer covering 
Ontario and national politics in print and 
television. 

• John Sciarra is the former director of oper- 
ations in my Office. He was instrumental in 
leading the implementation of our advertising 
review function and in drafting the guidelines 
that help ministries comply with the Act. 

These advisers provided invaluable assistance 
in our review of government advertising this past 
fiscal year. 



Advertising Review Activity, 
2010/11 



RESULTS OF OUR REVIEWS 

During the 2010/11 fiscal year, we reviewed 1,082 
individual advertising items in 165 submissions, 
with a total value of $50 million. This compares to 
159 submissions, comprising 600 individual ads, 
with a value of more than $40 million last year. 

We gave our decision in all cases within the 
required seven business days. The length of time 
required for a review and decision can vary, 
depending on the complexity of the ad and on the 
other work priorities of our review panel. Neverthe- 
less, average turnaround time during the past fiscal 
year was 3.5 business days. 

We also received and examined 22 pre-review 
submissions at a preliminary stage of development. 
Because pre-reviews are strictly voluntary on our 
part and outside the statutory requirements of the 
Act, they are second in priority to finished items. 
We nonetheless make every effort to complete them 
within a reasonable time. The average turnaround 
time for pre-review submissions in the 2010/11 fis- 
cal year was 5.9 business days. 

Of all the final submissions received in the 
2010/11 fiscal year, we rejected two: 

• A newspaper advertising campaign relating to 
the 10% rebate on electricity bills under the 
Ontario Clean Energy Benefit was rejected on 
the grounds that its primary objective was to 
foster a positive impression of the government 
party, contrary to section 6(1)5 of the Act. 
After reworking the campaign, the Ministry of 
Energy resubmitted it and we approved it. 
A template for a newspaper ad announcing 
"expanded" diabetes programs across the 
province was rejected primarily for failing to 
provide evidence that the program had in fact 
been expanded in every location. The Ministry 
of Health and Long-Term Care did not resub- 
mit the ad. 
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We also rescinded previously granted approval 
for three other digital video and television ads that 
were part of a campaign on medical wait times from 
the Ministry of Health and Long-Term Care after 
the Liberal Party of Ontario released an ad with 
strikingly similar visuals on the same subject. 

We also noted two contraventions of the Act — 
advertisements that ran without having first been 
submitted to us for review, as follows: 

• For the second straight year, the Ontario 
Provincial Police, which is overseen by the 
Ministry of Community Safety and Correc- 
tional Services, ran ads that it had not first 
submitted for review. The OPP advised that 
the contraventions were due to a lack of fam- 
iliarity with the Act. We determined that had 
these six ads been submitted, they would have 
been approved. 

• More than 100 ads in various media relating 
to Huronia Historical Parks and Fort William 
Historical Park ran without first having been 
submitted for review. These attractions are 
part of the Ministry of Tourism and Culture 
and thus covered by the Act. The Ministry 
advised us it has taken steps to ensure that the 
parks will submit all future ads for review. The 
ads that ran in contravention of the Act would 
likely have been approved if they had been 
submitted for review. 

As well, we also had serious concerns on a 
number of pre-review submissions. In almost all 
instances, these were revised, resubmitted, and 
subsequently approved. 

OTHER MATTERS 

Election Timing 

We noted in our 2007 Annual Report that the deci- 
sion to hold provincial elections on fixed dates 
every four years made it "important to consider 
how publicly funded government advertising 
should be dealt with in a pre-election period." 



At the time, we warned that "noticeable changes 
in the character, content, emphasis, or volume 
of government advertising in the period before a 
general election may be perceived as giving the 
governing party an advantage," and added that we 
"would consider not only the content of each adver- 
tising item, but also the current political circum- 
stances and the timing of the planned publication 
or dissemination of the item." 

We found issues with a few submissions in the 
months leading up to the most recent election on 
October 6, 2011 — for example, the Ministry of 
Health and Long-Term Care failed to provide the 
required notification of its plans to buy additional 
air time for previously approved radio and tele- 
vision ads. 

We also rejected two submissions and cited a 
third in violation of the Act in the months leading 
up to the election, as follows: 

• We rejected a radio campaign promoting the 
availability of free vaccinations for the rota- 
virus because it loosely resembled a Liberal 
Party of Ontario commercial on the medical 
screening of newborn infants. The Ministry of 
Health and Long-Term Care reworked the ad 
and resubmitted it, and we approved it. 

• We rejected a print and radio campaign about 
agricultural risk-management programs 
because it violated section 6(1)5 of the Act, 
which says that an ad must not have as a pri- 
mary objective to "foster a positive impression 
of the governing party." After quickly rework- 
ing the campaign, the Ministry of Agriculture, 
Food and Rural Affairs resubmitted it and we 
approved it. 

• We found an approved Ministry of Finance 
campaign on Ontario Savings Bonds in viola- 
tion after the first-click web page promoted in 
the ad contained a reference to the "McGuinty 
government," in contravention of section 
6(1)3 of the Act, which says ads "must not 
include the name, voice or image of a member 
of the Executive Council or a member of the 
Assembly." The Ministry quickly corrected it. 
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We would also like to point out a possible limita- 
tion to the Act relating to a mail insert from the 
Ministry of Energy that was included with electri- 
city bills just prior to the election. The insert touted 
the Ontario Clean Energy Benefit, the government's 
10% reduction on electricity bills for the next five 
years. We understand that utilities were required by 
the Ministry to include these inserts. Such inserts 
are not subject to the Act, which covers only unad- 
dressed bulk mail. However, we were concerned 
that the inserts, some of which arrived in mailboxes 
less than a month before the provincial election, 
could be seen as violating the intent of the Act. The 
insert likely would not have passed our review if 
it had been submitted to us. We expressed similar 
concerns last year regarding an insert included with 
HST rebate cheques. Both these examples highlight 
a possible limitation of the Act with respect to such 
inserts. 

Internet Advertising 

In our 2010 Annual Report, we cited an instance of 
a government ad running on-line that was similar 
to an ad we identified in 2009 as not meeting the 
required standards of the Act. We noted at the time 
that this underscored the limitations of the Act, 
which does not cover Internet advertising, a fast- 
growing segment of the advertising market. 

In the 2010/11 fiscal year, we noted that many 
reviewable ad campaigns included Internet com- 
ponents, some of which could have been found in 
violation of the Act had they been subject to our 
review. With the total value and number of Internet 
ads continuing to grow, the government should 
consider whether including Internet ads in the Act 
warrants consideration. 

Ministry of Infrastructure 

In September 2009, the then Ministry of Energy 
and Infrastructure submitted a television ad, called 
"Connects," for pre-review. We indicated at the time 
that the ad, which dealt with the merits of infra- 



structure, would likely not meet the standards of 
the Act because it contained little information and 
appeared self-congratulatory. The Ministry did not 
revise or resubmit the ad. 

In May 2011, we learned that Infrastructure 
Ontario, a provincial Crown corporation not gov- 
erned by the Act, intended to launch a television 
campaign using an ad very much like the "Con- 
nects" one we had rejected almost two years earlier. 

We advised the Ministry of Infrastructure of 
our concern that the Ministry may have knowingly 
allowed one of its agencies — an agency that reports 
to the Minister of Infrastructure — to run advertising 
very similar to advertising that had already been 
submitted in 2009 for pre-review and that had been 
found by my Office to have failed to meet the stan- 
dards of the Act. 

In response, a senior official of Infrastructure 
Ontario advised us that his staff were unaware 
of the 2009 version of the ad, prepared by the 
same agency that created the 2011 version. They 
expressed dissatisfaction with the fact that the ad 
agency was not more forthright in letting them 
know it was a second attempt at airing essentially 
the same ad that had been turned down previously. 

Our Office was informed by Infrastructure 
Ontario in September 2011 that the ad had not run. 



Expenditures on 
Advertisements and Printed 
Matter 



The Auditor General Act requires that the Auditor 
General report annually to the Legislative Assembly 
on expenditures for advertisements, printed mat- 
ter, and messages that are reviewable under the 
Government Advertising Act, 2004. 

Figure 1 contains expenditure details of individ- 
ual advertising campaigns reported to us by each 
ministry for media-buy costs; agency creative costs; 
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third-party production, talent, and distribution costs; 
and other third-party costs, such as translation. 

In order to test the completeness and accuracy 
of the reported advertising expenditures, my Office 
reviewed randomly selected payments to suppli- 
ers of advertising and creative services and their 
supporting documentation at selected ministries. 
We also performed certain compliance procedures 
with respect to the requirements of sections 2, 3, 
4, and 8 of the Government Advertising Act, 2004, 
which pertain to submission requirements and 
prohibition on the use of items pending the Auditor 
General's review. We found no matters of concern 
in our review work. 
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Figure 1: Expenditures for Reviewable Advertisements and Printed Matter under the 
Government Advertising Act, 2004, April 1, 2010-March 31, 2011 

Source of data: Ontario government offices 





#of 


mm 


Third-party Costs ($) 


Ministry/Campaign Title 


Submissions 


Items 


Agency Fees 


Production 


Talent 


Bulk Mail 


Other 


Agriculture, Food and Rural Affairs 


Foodland Ontario 2 


4 


51 


173,675 


989,886 


55,334 




228 


Invest in Ontario 2 


1 


2 












Ontario's Bio Advantage 


1 


1 












Pick Ontario Freshness 1 
















Royal Winter Fair 


1 


1 












Cabinet Office 


Institute of Public Administration of 
Canada 


1 


1 












Children and Youth Services 


Ontario Child Benefit 


2 


19 


31,900 


27,910 


22,500 




5,700 


Citizenship and Immigration 


Global Experience Ontario 


1 


1 












Order of Ontario 


1 


23 




5,386 








Remembrance Day 


1 


3 




946 








Community Safety and Correctional Services 


OPP 4 




6 












Public Notice - Security Industry 
Workers 1 
















RIDE 


2 


2 






14,427 






RIDE 1 








78 








Economic Development and Trade 


Business Immigration 


1 


10 


5,143 


7,000 






2,815 


Domestic Business Programs 


1 


4 


241,485 


388,253 


95,100 




1,820 


Go North 1 
















Go North 2 


6 


24 


189,508 


30,186 






9,507 


Invest Ontario 1 
















Invest Ontario 2 


15 


92 


846,563 


153,959 


56,942 




26,970 


Ontario Exports 


3 


10 


12,325 


3,475 






1,739 


Ontario Exports 1 
















Education 


Education Summit 


1 


2 












Full-day Kindergarten 2 


7 


62 


249,076 


361,366 


148,165 




16,833 


Kidstreet 


1 


1 












Speak Up 


1 


1 













1. ad submission from 2009/10, with more expenditures in 2010/11 

2. ad submission from 2010/11, with more expenditures in 2011/12 
4. contravention-ad was not submitted for review 

9. negative total due to media credits being applied 
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Media Costs ($) 


Ad Value* 


Campaign 


TV 


Radio 


Print 


Out-of-Home* 


($) 


Total ($) 




2,865,421 


528,854 




341,825 




4,955,223 






4,961 






4,961 










3,500 


3,500 




1,885 








1,885 










2,850 


2,850 












4,000 


4,000 






45,784 


244,191 






377,985 








1,250 






1,250 






141,019 






146,405 






26,576 






27,522 








330 






330 






61,366 






61,366 


256,949 






19,642 




291,018 


-2,510 










-2,432 9 








199,569 






214,527 


834,721 




86,241 






1,647,620 






45,465 






45,465 






625,509 




3,740 


858,450 


-34,175 




-21,770 


-17,284 


9,600 


-63,629 9 


2,062,017 




3,398,137 


598,445 


4,060 


7,147,093 






2,078 






19,617 






42,628 




5,000 


47,628 












10,821 


10,821 


3,558,749 




279,748 






4,613,937 










1,413 


1,413 










30,000 


30,000 



* Out-of-Home advertising includes, for example, billboards and transit posters. 

t Ad Value denotes the value of an ad space provided to government offices at no cost, often where the government has provided funding for a related event/ 
publication. 
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#of 


#of 


Third-party Costs ($] 






Submissions 


Items 


Agency Fees 


Production 


Talent 




Bulk Mail 


Other 


Energy and Infrastructure 


Infrastructure 1 
















Long-term Energy Plan 2 


3 


11 


122,527 


463,177 


1,364 




368,219 


Long-term Energy Plan 3 


1 


4 












Environment 


Public Notice - Air Standards 


2 


3 




575 








Public Notice - Environmental 
Sampling 


1 


1 




375 








Finance 


Children's Activity Tax Credit 


4 


61 


198,143 


159,934 


12,430 




19,492 


Ontario Budget 2 


2 


24 


42,848 


42,558 








Ontario Savings Bonds 


2 


37 


108,936 


95,096 


75,340 




10,385 


Government Services 


1-888-Business Info Line 




4 




6,007 






2,048 


Smartmoves 1 








4,900 






256 


Taking the Lead - ServiceOntario 




2 










4 


Taking the Lead - ServiceOntario 1 














2,162 


Validation Sticker Renewal 




19 


15,600 


9,200 






2,682 


Health and Long-Term Care 


Diabetes Programs 3 




1 












Health Care Connect 




2 


2,013 


9,431 






775 


Health Care Options 2 


13 


54 


122,194 


91,540 






56,149 


Health Care Options 6 




12 


25,046 


1,074,554 


65,093 


1,205 


9,801 


HealthForceOntario 8 




1 












MedsCheck for Diabetes Patients 




2 




10,944 


1,435 


2,100 


731 


Public Notice - Diabetes 1 
















Public Notice - Northern Health Care 
Services Co-ordination 


1 


6 












Seasonal Flu 


5 


68 


20,528 


20,158 


12,062 




4,039 


Stand Up to Diabetes 


4 


50 




20,400 








Health Promotion and Sport 


Diabetes 2 


2 


3 


175,628 


172,230 






1,995 


EatRight Ontario 1 
















Healthy Living 


1 


2 


333,955 


454,743 








World Junior Baseball 2 


1 


1 


1,105 










Labour 


Employment Standards 


1 


10 




26,837 






2,962 


Falls Prevention 5 


2 


15 













1. ad submission from 2009/10, with more expenditures in 2010/11 

2. ad submission from 2010/11, with more expenditures in 2011/12 

3. violation-ad was reviewed and did not meet the required standards 

5. costs incurred by WSIB 

6. approval withdrawn (see the "Results of Our Reviews" section earlier in this chapter) 

8. costs incurred by HealthForceOntario 

9. negative total due to media credits being applied 



Review of Government Advertising 





Media Costs ($) 


Ad Value* 


Campaign 


TV 




Radio 


Print 


Out-of-Home* 




T«lol IC\ 
















-1,513 










-1,513 9 


899,009 


644,462 








2,498,758 




















2,853 






3,428 






2,588 






2,963 






577,004 


582,396 


168,810 




1,718,209 






62,094 






147,500 


798,725 


145,687 


499,562 


276,209 




2,009,940 








71,667 






79,722 






19,586 






24,742 






3,980 






3,984 






369,497 






371,659 






37,723 






65,205 




















104,083 






116,302 




10,157 


1,632,489 


263,488 




2,176,017 


2,567,247 










3,742,946 
















562,330 








577,540 






311,891 






311,891 






19,053 






19,053 




1,366,943 




491,029 




1,914,759 






999,534 




3,250 


1,023,184 














349,853 






5,766 






5,766 


1,057,934 










1,846,632 










650 


1,755 








129,503 






159,302 















* Out-of-Home advertising includes, for example, billboards and transit posters. 

t Ad Value denotes the value of an ad space provided to government offices at no cost, often where the government has provided funding for a related event/ 
publication. 
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#of 


#of 


Third-party Costs ($) 








Submissions 


Items 


Agency Fees 


Production 


Talent 


1 


Bulk Mail 


Other 


Labour (continued) 


Jobs Protection Office 


1 


2 




2,500 








Safe at Work Ontario 


1 


1 




1,978 








Municipal Affairs and Housing 


Public Notice - Provincial Policy 
Statement Review 


1 


2 




632 






110 


Natural Resources 


Bear Wise 


1 


Q 

o 




DDU 








Bear Wise 1 
















Chronic Wasting Disease 


1 


1 












FireSmart Wildfire Prevention 1 








100 








FireSmart Wildfire Prevention 2 


2 


30 




685 






6 


Forest Resource Management 1 
















Kids Fish Art Contest 


1 


1 




100 








Lake Ontario Atlantic Salmon 


1 


1 




375 










Restoration 














Land Management 2 


8 


8 












Ontario Parks 1 
















Ontario Parks 2 


15 


17 












Outdoors Card 


1 


1 




150 








Species at Risk 


2 


2 




356 








Youth Employment 


1 


1 




175 








Northern Development, Mines and Forestry 


GO North 7 
















Northern Ontario Energy Credit 2 


3 


11 


275,625 


267,793 






7,500 


Northern Ontario Heritage Credit 2 


2 


4 












Public Notice - Forest Tenure and 


1 


2 




2,300 










Pricing Review 
















Research and Innovation 


Invest Ontario 7 
















Revenue 


Comprehensive Tax Reform 1 






3,696 










Ontario's Tax Plan 


9 


55 


106,015 


51,680 


27,990 


589,803 


45,705 


Tax Credits 2 


3 


40 


344,247 


174,637 


16,611 






Tourism and Culture 


Fort William Historical Park 4 




49 












Huronia Historical Parks 4 




67 




5,540 








Training, Colleges and Universities 


Employment Ontario 1 

















1. ad submission from 2009/10, with more expenditures in 2010/11 

2. ad submission from 2010/11, with more expenditures in 2011/12 
4. contravention-ad was not submitted for review 

7. ad developed by another ministry, but used by this ministry 



Review of Government Advertising 





Media Costs ($) 


Ad Value* 


Campaign 


TV 


Radio 


Print 


Out-of-Home* 


($) 


Total ($) 








2,125 






4,625 










2,490 


4,468 








23,403 






24,145 




on a 




OK 117 






1 1 O 01 K 






QQ QQQ 






QQ QQQ 






1,369 






1,369 






1,693 






1,793 






699 






1,390 






916 






916 










8,350 


8,450 










11,140 


11,515 






3,609 




2,775 


6,384 






7,516 






7,516 






17,190 






17,190 










8,350 


8,500 






34,284 






34,640 










7,300 


7,475 








1,966 






1,966 


121,929 


122,984 


111,261 






907,092 






1,290 




13,355 


14,645 






30,978 






33,278 












21,900 


21,900 






531,681 


716,201 






1,251,578 




935,740 


1,114,004 






2,870,937 












535,495 






27,448 


109,125 


182,462 




319,035 


44,955 


16,663 


58,150 


2,700 


350 


128,358 












4,250 


4,250 



* Out-of-Home advertising includes, for example, billboards and transit posters. 

t Ad Value denotes the value of an ad space provided to government offices at no cost, often where the government has provided funding for a related event/ 
publication. 
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#of 


#of 




Third-party Costs ($) 








Items 


Agency Fees 


Production 


Talent 1 


Bulk Mail 


Other 


Training, Colleges and Universities (continued) 


International Education 


1 


1 












Postsecondary Awareness & Public 
Education 


2 


66 


325,141 


625,014 


56,966 




21,801 


Study in Ontario 


1 


1 










1,805 


Transportation 


Veterans Graphic Licence Plates 


1 


2 


4,293 


2,008 


11,856 






Total 


165 


1,082 


3,977,215 


5,767,777 


673,615 


593,108 


624,239 



1. ad submission from 2009/10, with more expenditures in 2010/11 



Review of Government Advertising 





Media Costs ($) 




Ad Value* 


Campaign 


TV 


Radio 


Print 


Out-of-Home* 






















5,391 






5,391 


1,289,855 




791,174 


300,283 




3,410,234 






4,545 






6,350 




556,639 




10,393 






585,189 


16,896,404 


5,517,622 


13,227,945 


2,630,605 


159,144 





* Out-of-Home advertising includes, for example, billboards and transit posters. 

t Ad Value denotes the value of an ad space provided to government offices at no cost, often where the government has provided funding for a related event/ 
publication. 



Chapter 6 



The Standing Committee 
on Public Accounts 



Appointment and 
Composition of the 
Committee 




Members of the Standing Committee on Public 
Accounts (Committee) are appointed under the 
Standing Orders of the Legislative Assembly. The 
number of members from any given party reflects 
that party's representation in the Legislative Assem- 
bly. All members except the Chair are entitled to 
vote on motions, while the Chair may only vote to 
break a tie. The Committee is established for the 
duration of the Parliament, from the opening of its 
first session immediately following a general elec- 
tion to its dissolution. 

In accordance with the Standing Orders, a 
Standing Committee on Public Accounts was 
appointed on December 10, 2007, for the duration 
of the 39th Parliament. The membership of the 
Committee as of September 2011 was as follows: 

Norm Sterling, Chair, Progressive Conservative 

Peter Shurman, Vice-chair, Progressive 
Conservative 

Wayne Arthurs, Liberal 

Aileen Carroll, Liberal 

France Gelinas, New Democrat 

Jerry Ouellette, Progressive Conservative 

David Ramsay, Liberal 

Liz Sandals, Liberal 

David Zimmer, Liberal 



Role of the Committe 



The Committee examines, assesses, and reports 
to the Legislative Assembly on a number of issues. 
These include the economy and efficiency of 
government and broader-public-sector operations, 
the effectiveness of programs in achieving their 
objectives, and any issues that arise with respect to 
Ontario's Public Accounts. The Committee holds 
a number of hearings throughout the year relat- 
ing to matters raised in our Annual Report or our 
special reports and presents its observations and 
recommendations to the Legislative Assembly. 
Under sections 16 and 17 of the Auditor General Act, 
the Committee may also request that the Auditor 
General examine any matter in respect of the Public 
Accounts or undertake a special assignment on its 
behalf. 

AUDITOR GENERAL'S ADVISORY ROLE 
WITH THE COMMITTEE 

In accordance with section 16 of the Auditor Gen- 
eral Act, the Auditor General and senior staff attend 
committee meetings to assist the Committee with 
its reviews and its hearings relating to our Annual 
Report and Ontario's Public Accounts. 
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The Standing Committee on Public Accounts 



Committee Procedures and 
Operations 



The Committee may meet weekly when the Legisla- 
tive Assembly is sitting, and, with the approval of 
the House, at any other time of its choosing. All 
meetings are open to the public except for those 
dealing with the Committee's agenda or the prep- 
aration of its reports. All public committee proceed- 
ings are recorded in Hansard, the official verbatim 
report of government debates, speeches, and other 
Legislative Assembly proceedings. 

The Committee identifies matters of interest 
from our Annual Report or our special reports and 
conducts hearings on them. It typically focuses on 
reports from the value-for-money chapter of our 
Annual Report for review. In recent years, each of 
the three political parties has selected three audits 
or other sections from our Annual Report to hold 
hearings on. The Committee also considers whether 
to hold hearings on any special reports we have 
tabled during the year. 

At each hearing, the Auditor General, along with 
the Committee's researcher, briefs the Committee 
on the applicable report section and the responses 
to our findings and recommendations from the min- 
istry, Crown agency, or organization in the broader 
public sector that was the subject of the audit. The 
Committee then requests that senior officials from 
the auditee(s) appear at the hearing and respond 
to questions from committee members. Because 
our Annual Report deals with operational, admin- 
istrative, and financial rather than policy matters, 
ministers are rarely requested to attend. Once its 
hearings are completed, the Committee reports its 
comments and recommendations to the Legislative 
Assembly. 

The Clerk of the Committee also requests that 
those auditees that were not selected for hearings 
update the Committee on what actions they are tak- 
ing to address the concerns raised in our reports. 



MEETINGS HELD 

The Committee met 19 times during the October 
2010- June 2011 period. Some of these meetings 
were held to complete Committee reports about the 
hearings on sections from both our 2009 and 2010 
Annual Reports as well as on our October 2009 
Special Report on Ontario's Electronic Health Records 
Initiative. 

Four value-for-money audit sections and two 
follow-up sections from our 2010 Annual Report 
were chosen for hearings in 2011, as follows: 

• 3.03-Family Responsibility Office; 

• 3.07-Infrastructure Stimulus Spending; 

• 3.08-Municipal Property Assessment 
Corporation; 

• 3.09-Non-hazardous Waste Disposal and 
Diversion; 

• 4.05-Commercial Vehicle Safety and Enforce- 
ment Program; and 

• 4.11-Hospital Board Governance. 

REPORTS 0FTHE COMMITTEE 

The Committee issues reports and letters on its 
work for tabling in the Legislative Assembly. These 
reports and letters summarize the information 
gathered by the Committee during its meetings and 
include the Committee's comments and recommen- 
dations. Once tabled, all committee reports and let- 
ters are publicly available through the Clerk of the 
Committee or on-line at www.ontla.on.ca. 

Committee reports typically include recommen- 
dations and request that management of the min- 
istry, agency, or broader-public-sector organization 
provide the Committee Clerk with responses within 
a stipulated time frame. Our Office reviews these 
responses, and, in any subsequent audits of that 
operational area, we take the Committee's recom- 
mendations into consideration. 

During the period from October 2010 through 
June 2011, the Committee tabled the following 17 
reports stemming from hearings held on sections 
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from our 2009 and 2010 Annual Reports (AR) as 
well as on one of our special reports: 

• Unfunded Liability of the Workplace Safety 
and Insurance Board (Section 3.14, 2009 AR) 
(tabled on October 5, 2010); 

• Unspent Grants (Chapter 2, 2009 AR) (tabled 
on October 20, 2010); 

• Teletriage Health Services (Section 3.13, 2009 
AR) (tabled on October 26, 2010); 

• Bridge Inspection and Maintenance (Section 
3.02, 2009 AR) (tabled on November 1, 
2010); 

• Ontario's Electronic Health Records (October 
2009 Special Report) (tabled on November 4, 
2010); 

• Ontario Disability Support Program (Section 
3.09, 2009 AR) (tabled on November 24, 
2010); 

• Education Quality and Accountability Office 
(Section 3.04, 2009 AR) (tabled on November 
25,2010); 

• Literacy and Numeracy Secretariat (Section 
3.07, 2009 AR) (tabled on December 6, 2010); 

• Infection Prevention and Control at Long-term- 
care Homes (Section 3.06, 2009 AR) (tabled 
on February 28, 2011); 

• Assistive Devices Program (Section 3.01, 2009 
AR) (tabled on May 18, 2011); 

• Commercial Vehicle Safety and Enforcement 
Program (Section 4.05, 2010 AR) (tabled on 
May 18, 2011); 

• Infrastructure Stimulus Spending (Section 
3.07, 2010 AR) (tabled on May 18, 2011); 



• Hospital Board Governance (Section 4.11, 2010 
AR) (tabled on May 18, 2011); 

• Municipal Property Assessment Corporation 
(Section 3.08, 2010 AR) (tabled on May 30, 
2011); 

• Family Responsibility Office (Section 3.03, 
2010 AR) (tabled on May 30, 2011); 

• Non-hazardous Waste Disposal and Diversion 
(Section 3.09, 2010 AR) (tabled on May 30, 
2011); and 

• Public Accounts Committee Best Practice: Assist- 
ive Devices Program (Section 3.01, 2009 AR) 
(tabled on May 30, 2011). 

The last report described the Committee's pro- 
cess in following up on its earlier (May 18) report 
stemming from our 2009 audit of the Assistive 
Devices Program. 

CANADIAN COUNCIL OF PUBLIC 
ACCOUNTS COMMITTEES 

The Canadian Council of Public Accounts Commit- 
tees (CCPAC) consists of delegates from federal, 
provincial, and territorial public accounts com- 
mittees from across Canada. CCPAC holds a joint 
annual conference with the Canadian Council of 
Legislative Auditors to discuss issues of mutual 
interest. 

The 32nd annual conference was hosted by 
Nova Scotia and was held in Halifax from August 28 
to 30, 2011. 



Chapter 7 




The Office of the 
Auditor General of Ontario 



The Office of the Auditor General of Ontario 
(Office) serves the Legislative Assembly and the 
citizens of Ontario by conducting value-for-money 
and financial audits and reviews and reporting on 
them. By doing this, the Office helps the Legislative 
Assembly hold the government, its administrators, 
and grant recipients accountable for how prudently 
they spend public funds and for the value they 
obtain, on behalf of Ontario taxpayers, for the 
money spent. 

The work of the Office is performed under the 
authority of the Auditor General Act. In addition, 
under the Government Advertising Act, 2004, the 
Auditor General is responsible for reviewing and 
deciding whether or not to approve certain types of 
proposed government advertising (see Chapter 5 
for more details on the Office's advertising review 
function). Both acts can be found at www.e-laws. 
gov.on.ca. 

In an election year, the Auditor General is 
also required to review the reasonableness of the 
government's pre-election report on its expecta- 
tions for the financial performance of the province 
over the next three fiscal years. Because 2011 was 
an election year, the government issued its pre- 
election report on April 26, 2011, and the results of 
our review were released on June 28, 2011. 




General Overview 



VALUE-FOR-MONEY AUDITS IN THE 
ANNUAL REPORT 

About two-thirds of the Office's work relates to 
value-for-money auditing. The Office's value-for- 
money audits are assessments of how well a given 
"auditee" (the entity that we audit) manages and 
administers its programs or activities. The auditees 
that the Office has the authority to conduct value- 
for-money audits of are: 

• Ontario government ministries; 

• Crown agencies; 

• Crown-controlled corporations; and 

• organizations in the broader public sector 
that receive government grants (for example, 
agencies that provide mental-health services, 
children's aid societies, community colleges, 
hospitals, long-term-care homes, school 
boards, and universities) . 

The Auditor General Act (Act) [in subclauses 
12(2) (f) (iv) and (v)] identifies the criteria to be 
considered in this assessment: 

• Money should be spent with due regard for 
economy. 

• Money should be spent with due regard for 
efficiency. 

• Appropriate procedures should be in place to 
measure and report on the effectiveness of 
programs. 
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We also examine the level of service that is being 
provided to the public and, where possible, com- 
pare it to the best practices of other jurisdictions 
that deliver similar services. 

The Act requires that, if the Auditor General 
observes instances where the three value-for- 
money criteria have not been met, he or she report 
on them. The Act also requires that he or she report 
on instances where the following was observed: 

• Accounts were not properly kept or public 
money was not fully accounted for. 

• Essential records were not maintained or the 
rules and procedures applied were not suf- 
ficient to: 

• safeguard and control public property; 

• check effectively the assessment, collec- 
tion, and proper allocation of revenue; or 

• ensure that expenditures were made only 
as authorized. 

• Money was expended other than for the pur- 
poses for which it was appropriated. 

Assessing the extent to which the auditee was 
controlling against these risks is technically "com- 
pliance" audit work but is generally incorporated 
into both value-for-money audits and "attest" audits 
(discussed in a later section). Other compliance 
work that is typically included in our value-for- 
money audits is: 

• identifying the key provisions in legislation 
and the authorities that govern the auditee or 
the auditee's programs and activities as well 
as those that the auditee's management is 
responsible for administering; and 

• performing the tests and procedures we deem 
necessary to obtain reasonable assurance 
that the auditee's management has complied 
with these key legislation and authority 
requirements. 

Government programs and activities are the 
result of government policy decisions. Thus, we 
could say that our value-for-money audits focus 
on how well management is administering and 
executing government policy decisions. However, 
although we may provide information on the 



impacts of government policy, we do not opine on 
the merits of government policy. Rather, it is the 
Legislative Assembly that holds the government 
accountable for policy matters. The Legislative 
Assembly continually monitors and challenges gov- 
ernment policies through questions during legisla- 
tive sessions and through reviews of legislation and 
expenditure estimates. 

In planning, performing, and reporting on our 
value-for-money work, we follow the relevant 
professional standards established by the Canadian 
Institute of Chartered Accountants. These stan- 
dards require that we have processes for ensuring 
the quality, integrity, and value of our work. Some 
of the processes we use are described below. 

Selecting What to Audit 

The Office audits major ministry programs and 
activities at approximately five- to seven-year inter- 
vals. We do not audit organizations in the broader 
public sector and Crown-controlled corporations on 
the same cycle because there are such a great num- 
ber of them and their activities are so numerous 
and diverse. Since our mandate expanded in 2004 
to allow us to audit these auditees, our audits have 
covered a wide range of topics across a broad range 
of sectors, including health (hospitals, long-term- 
care homes, and mental-health service providers), 
education (school boards, universities, and col- 
leges), and social services (children's aid societies 
and social service agencies), as well as several large 
Crown-controlled corporations. 

In selecting what program, activity, or organiza- 
tion to audit each year, we consider how great the 
risk is that an auditee is not delivering public ser- 
vices in a cost-effective manner. To help us choose 
higher-risk audits, we consider factors such as: 

• the results of previous audits and related 
follow-ups; 

• the total revenues or expenditures involved; 

• the impact of the program, activity, or organ- 
ization on the public; 



• the complexity and diversity of the auditee's 
operations; 

• recent significant changes in the auditee's 
operations; and 

• the significance of the issues an audit might 
identify. 

Another factor we take into account in the selec- 
tion process is what work the auditee's internal 
auditors have completed or planned. Depending on 
what that work consists of, we may defer an audit 
or change our audit's scope to avoid duplication of 
effort. In other cases, we do not diminish the scope 
of our audit but rely on and present the results of 
internal audit work in our audit report. 

Setting Audit Objectives, Audit Criteria, 
and Assurance Levels 

When we begin an audit, we set an objective for 
what we want to achieve. We then develop suitable 
audit criteria that cover the key systems, poli- 
cies, and procedures that should be in place and 
operating effectively. Developing criteria involves 
extensively researching sources such as recognized 
bodies of experts; other bodies or jurisdictions 
delivering similar programs and services; manage- 
ment's own policies and procedures; applicable cri- 
teria successfully applied in other audits or reviews; 
and applicable laws, regulations, and other author- 
ities. To further ensure their suitability, the criteria 
we develop are discussed with the senior manage- 
ment responsible for the program or activity at the 
planning stage of the audit. 

The next step is designing and conducting tests 
and procedures to address our audit objective 
and criteria, so that we can reach conclusions on 
them. Each audit report has a section titled "Audit 
Objective and Scope," in which the audit objective 
is stated. 

Conducting tests and procedures to gather 
information has its limitations. We therefore cannot 
provide what is called an "absolute level of assur- 
ance" that our audit work identifies all significant 
matters. Other factors also contribute to this. For 
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example, we may conclude that the auditee had a 
control system in place for a process or procedure 
that was working effectively to prevent a particular 
problem from occurring; but auditee management 
or staff are often able to circumvent such control 
systems, so we cannot guarantee that the prob- 
lem will never arise. Also, much of the evidence 
available for concluding on our objective is more 
persuasive than it is conclusive, and we must rely 
on professional judgment in much of our work — for 
example, in interpreting information. 

For all these reasons, the assurance that we plan 
for our work to provide is at an "audit level" — the 
highest reasonable level of assurance that we can 
obtain using our regular audit procedures. Spe- 
cifically, an audit level of assurance is obtained by 
interviewing management and analyzing the infor- 
mation it provides; examining and testing systems, 
procedures, and transactions; confirming facts 
with independent sources; and, where necessary 
because we are examining a highly technical area, 
obtaining expert assistance and advice. 

With respect to the information that manage- 
ment provides, under the Act we are entitled to 
have access to all relevant information and records 
necessary to the performance of our duties. Out of 
respect for the principle of Cabinet privilege, we 
do not seek access to the deliberations of Cabinet. 
However, the Office can access virtually all other 
information contained in Cabinet submissions or 
decisions that we deem necessary to fulfill our 
responsibilities under the Act. 

Infrequently, the Office will perform a review 
rather than an audit. A review provides a moder- 
ate level of assurance, obtained primarily through 
inquiries and discussions with management; analy- 
ses of information that management provides; and 
only limited examination and testing of systems, 
procedures, and transactions. We perform reviews 
when, for example, providing a higher level of 
assurance has prohibitive costs or is unnecessary, or 
other factors relating to the nature of the program 
or activity make a review more appropriate than an 
audit. This year, we conducted a review, contained 
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in Chapter 3 of this report, of the electricity sector's 
stranded debt. In the 2009 audit year, we conducted 
a review of the unfunded liability of the Workplace 
Safety and Insurance Board, which was well received 
by the Standing Committee on Public Accounts. 

Communicating with Management 

To help ensure the factual accuracy of our observa- 
tions and conclusions, staff from our Office com- 
municate with the auditee's senior management 
throughout the value-for-money audit or the review. 
Before beginning the work, our staff meet with 
management to discuss the objective and criteria 
and the focus of our work in general terms. During 
the audit or review, our staff meet with manage- 
ment to review progress and ensure open lines of 
communication. At the conclusion of on-site work, 
management is briefed on the preliminary results 
of the work. A draft report is then prepared and dis- 
cussed with the auditee's senior management. The 
auditee's management provides written responses to 
our recommendations, and these are discussed and 
incorporated into the draft report. The Auditor Gen- 
eral finalizes the draft report (on which the Chap- 
ter 3 section of the Annual Report will be based) 
with the deputy minister or head of the agency, 
corporation, or grant-recipient organization, after 
which the report is published in the Annual Report. 

SPECIAL REPORTS 

As required by the Act, the Office reports on its aud- 
its in an Annual Report to the Legislative Assembly. 
In addition, the Office may make a special report to 
the Legislative Assembly at any time, on any matter 
that, in the opinion of the Auditor General, should 
not be deferred until the Annual Report. 

Two sections of the Act authorize the Auditor 
General to undertake additional special work. 
Under section 16, the Standing Committee on Pub- 
lic Accounts may resolve that the Auditor General 
must examine and report on any matter respecting 
the Public Accounts. Under section 17, the Legisla- 



tive Assembly, the Standing Committee on Public 
Accounts, or a minister of the Crown may request 
that the Auditor General undertake a special assign- 
ment. However, these special assignments are not 
to take precedence over the Auditor General's other 
duties, and the Auditor General can decline such 
an assignment requested by a minister if he or she 
believes that it conflicts with other duties. 

In recent years, when we have received a special 
request under section 16 or 17, our normal practice 
has been to obtain the requester's agreement that 
the special report will be tabled in the Legislature 
on completion and made public at that time. 

Over the past five years, we have issued eight 
special reports, as well as two reports reviewing 
the government's pre-election report on Ontario's 
finances. No special audits were requested during 
the 2010/11 audit year. 

ATTEST AUDITS 

Attest audits are examinations of an auditee's 
financial statements. In such audits, the auditor 
expresses his or her opinion on whether the finan- 
cial statements present information on the auditee's 
operations and financial position in a way that 
is fair and that complies with certain accounting 
policies (in most cases, with Canadian generally 
accepted accounting principles) . As mentioned in 
the overview of value-for-money audits, compliance 
audit work is often incorporated into attest audit 
work. Specifically, we assess the controls for man- 
aging risks relating to improperly kept accounts; 
unaccounted-for public money; lack of recordkeep- 
ing; inadequate safeguarding of public property; 
deficient procedures for assessing, collecting, 
and properly allocating revenue; unauthorized 
expenditures; and not spending money on what it is 
intended for. 

The Auditees 

Every year, we audit the consolidated financial 
statements of the province and the accounts of 
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many agencies of the Crown. Specifically, the Act 
[in subsections 9(1), (2), and (3)] requires that: 

• the Auditor General audit the accounts and 
records of the receipt and disbursement of 
public money forming part of the province's 
Consolidated Revenue Fund, whether held in 
trust or otherwise; 

• the Auditor General audit the financial state- 
ments of those agencies of the Crown that are 
not audited by another auditor; 

• public accounting firms that are appointed 
auditors of certain agencies of the Crown 
perform their audits under the direction of the 
Auditor General and report their results to the 
Auditor General; and 

• public accounting firms auditing Crown- 
controlled corporations deliver to the Auditor 
General a copy of the audited financial state- 
ments of the corporation and a copy of the 
accounting firm's report of its findings and 
recommendations to management (typically 
contained in a management letter) . 

Chapter 2 discusses this year's attest audit of the 
province's consolidated financial statements. 

We do not generally discuss the results of attest 
audits of agencies and Crown-controlled corporations 
in this report. Agency legislation normally stipulates 
that the Auditor General's reporting responsibil- 
ities are to the agency's board and the minister(s) 
responsible for the agency. Our Office also provides 
copies of our independent auditor's reports and of 
the related agency financial statements to the dep- 
uty minister of the associated ministry, as well as to 
the Secretary of the Treasury Board. 

Where an agency attest audit notes areas where 
management must make improvements, the auditor 
prepares a draft findings report and discusses it 
with senior management. The report is revised to 
reflect the results of that discussion. After the draft 
report is cleared and the agency's senior manage- 
ment responds to it in writing, the auditor prepares 
a final report, which is discussed with the agency's 
audit committee if one exists. If a matter were so 
significant that we felt it should be brought to the 



attention of the Legislature, we would include it in 
our Annual Report. 

Exhibit 1, Part 1 lists the agencies that were 
audited during the 2010/11 audit year. The Office 
currently contracts with public accounting firms 
to audit a number of these agencies on the Office's 
behalf. Exhibit 1, Part 2, and Exhibit 2 list the 
agencies of the Crown and the Crown-controlled 
corporations, respectively, that public accounting 
firms audited during the 2010/11 audit year. 

OTHER STIPULATIONS OF THE AUDITOR 
GENERAL ACT 

The Auditor General Act came about with the pas- 
sage, on November 22, 2004, of Bill 18, the Audit 
Statute Law Amendment Act, which received Royal 
Assent on November 30, 2004. The purpose of Bill 
18 was to make certain amendments to the Audit 
Act to enhance the ability of the Office to serve the 
Legislative Assembly. The most significant amend- 
ment contained in Bill 18 was the expansion of the 
Office's value-for-money audit mandate to organ- 
izations in the broader public sector that receive 
government grants. This 2011 AnnualReport marks 
the sixth year of our expanded audit mandate. 

Appointment of Auditor General 

Under the Act, the Auditor General is appointed as 
an officer of the Legislative Assembly by the Lieuten- 
ant Governor-in-Council — that is, the Lieutenant 
Governor appoints the Auditor General on and with 
the advice of the Executive Council (the Cabinet) . 
The appointment is made "on the address of the 
Assembly," meaning that the appointee must be 
approved by the Legislative Assembly. The Act also 
requires that the Chair of the Standing Commit- 
tee on Public Accounts — who, under the Standing 
Orders of the Legislative Assembly, is a member of 
the official opposition — be consulted before the 
appointment is made (for more information on the 
Committee, see Chapter 6). The last two Auditors 
General were selected through a formal competitive 



2011 Annual Report of the Office of the Auditor General of Ontario 



selection process, with members from each political 
party sitting on the selection committee. 

Independence 

The Auditor General and staff of the Office are 
independent of the government and its administra- 
tion. This independence is an essential safeguard 
that enables the Office to fulfill its auditing and 
reporting responsibilities objectively and fairly. 

The Auditor General is appointed to a 10-year, 
non-renewable term, and can be dismissed only for 
cause by the Legislative Assembly. Consequently, 
the Auditor General is able to maintain an arm's- 
length relationship with the government and the 
political parties in the Legislative Assembly and is 
thus free to fulfill the Office's legislated mandate 
without political pressure. 

The Board of Internal Economy (Board) — an 
all-party legislative committee that is independent 
of the government's administrative process — 
reviews and approves the Office's budget, which is 
subsequently laid before the Legislative Assembly. 
As required by the Act, the Office's expenditures 
relating to the 2010/11 fiscal year have been 
audited by a firm of chartered accountants, and 
the audited financial statements of the Office are 
submitted to the Board and subsequently must be 
tabled in the Legislative Assembly. The audited 
statements and related discussion of expenditures 
for the year are presented at the end of this chapter. 

CONFIDENTIALITY OF WORKING PAPERS 

In the course of our reporting activities, we prepare 
draft audit reports and findings reports that are 
considered to be an integral part of our audit work- 
ing papers. It should be noted that these working 
papers, according to section 19 of the Auditor Gen- 
eral Act, do not have to be laid before the Legislative 
Assembly or any of its committees. As well, because 
our Office is exempt from the Freedom of Informa- 
tion and Protection of Privacy Act, our draft reports 
and audit working papers, which include all infor- 



mation obtained during the course of an audit from 
the auditee, cannot be accessed from our Office, 
thus further ensuring confidentiality. 

CODE OF PROFESSIONAL CONDUCT 

The Office has a Code of Professional Conduct to 
encourage staff to maintain high professional stan- 
dards and ensure a professional work environment. 
The Code is intended to be a general statement of 
philosophy, principles, and rules regarding conduct 
for employees of the Office, who have a duty to 
conduct themselves in a professional manner and to 
strive to achieve the highest standards of behaviour, 
competence, and integrity in their work. 

The Code explains why these expectations exist 
and further describes the Office's responsibilities to 
the Legislative Assembly, the public, and our aud- 
itees. The Code also provides guidance on disclo- 
sure requirements and the steps to be taken to avoid 
conflict-of-interest situations. All employees are 
required to complete an annual conflict-of-interest 
declaration. 





Office Organization and 
Personnel 



The Office is organized into portfolio teams — a 
framework that groups together related audit enti- 
ties and fosters expertise in the various areas of 
government activity, such as health care, education, 
social services, and the environment. The port- 
folios, which are loosely based on the government's 
organization into ministries, are each headed by a 
Director, who oversees and is responsible for the 
audits within the assigned portfolio. Assisting the 
Directors and rounding out the teams are a number 
of Audit Managers and various other audit staff 
(see Figure 1). 

The Auditor General, the Deputy Auditor Gen- 
eral, the Directors, and the Managers of Human 
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Figure 1: Office Organization, September 30, 2011 



Auditor General 



Jim McCarter 



Deputy Auditor General 



Gary Peall 



Human Resources 



Annemarie Wiebe, Manager 
Shayna Whiteford 



Quality Assurance and Special Projects 



Paul Amodeo, Manager 
Kristin Snowden, Manager 
Shariq Saeed 



Standards and Research 



Rebecca Yosipovich, Manager 



Operations 



Administration 

Shanta Persaud 
Maureen Bissonnette 
Sohani Myers 
Christine Wu 

Communications and Government Advertising Review 

Christine Pedias, (Acting) Manager 
Mariana Green 
Shirley McGibbon 
Tiina Randoja 

Information Technology 

Peter Lee 
Shams Ali 



Audit Porfolios and Staff 



Community and Social Services, 
and Revenue 



Walter Bordne, Director 
Wendy Cumbo, Manager 
Nick Stavropoulos, Manager 
Johan Boer 
Dimitar Dimitrov 
Katrina Exaltacion 
Inna Guelfand 
Li-Lian Koh 



Jenna Lasky 
Michael Okulicz 
Shreya Shah 
Zachary Thomas 
Tiffany Yau 



Crown Agencies (1), Finance 



John McDowell, Director 
Walter Allan, Manager 
Tom Chatzidimos 
Kandy Fletcher 
Mary Romano 
Megan Sim 



Crown Agencies (2) 



Laura Bell, Director 
Teresa Carello, Manager 
Denise Young, Manager 
Margaret Chen 
Constantino De Sousa 
Roger Munroe 
Cynthia Tso 



Education and Training 



Gerard Fitzmaurice, Director 
Emanuel Tsikritsis, Manager 
Tara Beheshti 
Tino Bove 
Zahra Jaffer 
Rumi Janmohamed 
Mythili Kandasamy 



Michael Katsevman 
Nina Khant 
Mark Smith 
Ellen Tepelenas 
Dora Ulisse 



Health and Health Promotion 



Rudolph Chiu, Director 
Kim Cho, Manager 
Gigi Yip, Manager 
Bartosz Amerski 
Izabela Beben 
Ariane Chan 
Anita Cheung 
Helen Chow 
Marcia DeSouza 



Lisa Li 

Oscar Rodriguez 
Pasha Sidhu 
Alexander Truong 
Alia Volodina 



Health and Long-term-care Providers 



Susan Klein, Director 
Vanna Gotsis, Manager 
Naomi Herberg, Manager 
Kevin Aro 
Sally Chang 
Jennifer Fung 
Ingrid Goh 



Justin Hansis 
Veronica Ho 
Linde Qiu 



Justice and Regulatory 



Vince Mazzone, Director 
Rick MacNeil, Manager 
Fraser Rogers, Manager 
Vivian Sin, Manager 
Rashmeet Gill 
Kiran Grewal 
Tanmay Gupta 
Thomas loannou 
Alfred Kiang 



Margaret Lam 
Alice Nowak 
Brian Wanchuk 
Celia Yeung 



Public Accounts, Environment, and 
Natural Resources 



Gus Chagani, Director 
Sandy Chan, Manager 
Bill Pelow, Manager 
Allen Fung 
Mark Hancock 
Aldora Harrison 
Zhenya Stekovic 



Georgegiana Tanudjaja 
Janet Wan 
Jing Wang 
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Results Produced by the 
Office This Year 




Resources and of Communications and Govern- 
ment Advertising Review make up the Office's 
Senior Management Committee. 



Canadian Council of 
Legislative Auditors 




This year, Nova Scotia hosted the 39th annual 
meeting of the Canadian Council of Legislative 
Auditors (CCOLA) in Halifax, from August 28 to 
30, 2011. This annual gathering has, for the last 
32 years, been held jointly with the annual confer- 
ence of the Canadian Council of Public Accounts 
Committees (CCPAC) . It brings together legislative 
auditors and members of the Standing Committees 
on Public Accounts from the federal government 
and the provinces and territories, and provides 
a useful forum for sharing ideas and exchanging 
information. 



International Visitors 



As an acknowledged leader in value-for-money 
auditing, the Office often receives requests to meet 
with visitors and delegations from abroad to discuss 
the roles and responsibilities of the Office and to 
share our value-for-money and other audit experi- 
ences with them. During the audit year covered by 
this report, the Office met with legislators/public 
servants/auditors from Belize, Benin, Cameroon, 
China (both national and provincial audit offices), 
Ghana, Kenya, Pakistan (Punjab province), Saint 
Lucia, Tanzania, and Vietnam. 



The 2010/11 fiscal year was another successful year 
for the Office. In total, we completed 13 value-for- 
money audits and one review, and released a report 
on our statutory review of the 2011 Pre-Election 
Report on Ontario's Finances. This year our value- 
for-money audits in the broader public sector exam- 
ined aspects of a number of provincially funded 
agencies, such as the LCBO, the Ontario Trillium 
Foundation, Legal Aid Ontario, the Office of the 
Children's Lawyer, the Ontario Power Authority, the 
Ontario Energy Board, and the Financial Services 
Commission of Ontario (auto insurance regula- 
tion). Of these, only the LCBO had previously been 
subject to a value-for-money audit by our Office. 
In addition, we examined a variety of government 
activities also of importance to Ontarians, including 
renewable energy initiatives, student success initia- 
tives (including visits to selected school boards and 
schools), forest management, alternative funding 
arrangements for physicians, oversight of private 
career colleges, and supportive services provided to 
people with disabilities. 

As mentioned in the earlier "Attest Audits" 
section, we are responsible for auditing the prov- 
ince's consolidated financial statements (further 
discussed in Chapter 2), as well as the statements of 
more than 40 Crown agencies. We again met all of 
our key financial-statement audit deadlines while 
continuing our investment in training to success- 
fully implement significant revisions to accounting 
and assurance standards and methodology for 
conducting our financial-statement audits. 

We successfully met our review responsibilities 
under the Government Advertising Act, 2004, as 
further discussed in Chapter 5. 

The results produced by the Office this year 
would clearly not have been possible without the 
hard work and dedication of our staff, as well as the 
assistance of our contract staff and expert advisers. 
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As has been the case in recent years, with a number 
of senior staff retiring or on leave, contract staff 
were important to us again this year, and they filled 
in admirably. 



pensation increases for such professionals in the 
private sector. A more detailed discussion of the 
changes in our expenses and some of the challenges 
we are facing follows. 




The following discussion and our financial state- 
ments outline the Office's financial results for the 
2010/11 fiscal year. 

Figure 2 provides a comparison of our approved 
budget and expenditures over the last five years. 
Figure 3 presents the major components of our 
spending and shows that more than 74% (73% in 
2009/10) related to salary and benefit costs for our 
staff, while professional and other services and rent 
constituted most of the remainder. These propor- 
tions have been relatively stable in recent years. 

Overall, our expenses increased 2.1% (2.2% 
in 2009/10) and were again significantly under 
budget. Over the five-year period presented in 
Figure 2, we have returned unspent appropriations 
totalling $7.6 million. The main reason for this is 
that we have historically faced challenges in hiring 
and retaining qualified professional staff in the 
competitive Toronto job market — our public-service 
salary ranges have simply not kept pace with com- 



SALARIES AND BENEFITS 

Our salary costs rose 4.2% this year, while benefit 
costs were largely unchanged from the previous 
year. 

Given the legislated freeze on salary ranges, this 
increase related largely to promotional increases 
earned by trainees who obtained their professional 
accounting designations during the year and by 
those staff who demonstrated the ability to take on 
additional responsibilities. Many of our trainees 
earned their professional accounting designation 
during the year and remained with us. To be 
competitive, we must pay our newly qualified staff 
considerably more than they were paid as trainees, 
and salaries for qualified accountants rise fairly 
quickly in the first five years following qualification. 

With the economic uncertainty and the con- 
tinuing need for cost containment, we remained 
cautious about staffing up when staff departed, 
delaying the replacement of retiring senior staff 
and hiring experienced but more junior staff as 
opportunities arose. As a result, our average staff- 
ing over the course of this year was about the same 



Figure 2: Five-year Comparison of Spending (Accrual Basis) ($ 000) 
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H 2006/07 


2007/08 


2008/09 


2009/10 


2010/11 


Approved budget 


13,992 


15,308 


16,245 


16,224 


16,224 


Actual expenses 


salaries and benefits 


8,760 


9,999 


10,279 


10,862 


11,233 


professional and other services 


1,264 


1,525 


1,776 


1,489 


1,491 


rent 


985 


1,048 


1,051 


1,069 


1,036 


travel and communications 


363 


397 


332 


360 


337 


other 


930 


1,033 


1,096 


1,073 


1,071 


Total 


12,302 


14,002 


14,534 


14,853 


15,168 


Returned to province* 


1,730 


1,608 


1,561 


1,498 


1,223 



"These amounts are typically slightly different than the excess of appropriation over expenses as a result of non-cash expenses (such as amortization of capital 
assets and employee future benefit accruals). 
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Figure 3: Spending by Major Expenditure Category, 
2010/11 

Prepared by the Office of the Auditor General of Ontario 



other (7.1%) 



travel and 

communications (2.2%) 



Figure 4: Staffing, 2005/06-2010/11 

Prepared by the Office of the Auditor General of Ontario 

unfilled positions 
I I actual average FTEs 



rent (6 



professional and 
other services 
(9.8%) 



salaries and 
benefits (74.1%) 




as last year, at 104. After reaching as high as 110 
late last year and the early part of this year, staff 
departures have resumed as the market for profes- 
sional accountants has remained robust despite 
economic uncertainties. The growing complexity 
of our audits demands that we use highly qualified, 
experienced staff as much as possible. However, 
our hiring continues to be primarily at more junior 
levels, given that our salaries and benefits are 
competitive at these levels. We quickly fall behind 
private- and broader-public-sector salary scales for 
more experienced professional accountants. This 
is one reason that, as Figure 4 shows, we still have 
a number of unfilled positions. The challenge of 
maintaining and enhancing our capacity to perform 
these audits will only increase as more of our most 
experienced staff retire over the next few years. 

Under the Act, our salary levels must be compar- 
able to the salary ranges of similar positions in the 
government. These ranges remain uncompetitive 
with the salaries that both the not-for-profit and the 
private sectors offer. According to the most recent 
survey by the Canadian Institute of Chartered 
Accountants published in 2011, average salaries for 
CAs in government ($120,600) were 13% lower 
than those in the not-for-profit sector ($136,400) 



and, most importantly, 24% lower than those work- 
ing for professional service CA firms in Ontario 
($150,000), which are our primary competitors 
for professional accountants. The salaries of our 
highest-paid staff in the 2010 calendar year are 
disclosed in Note 7 to our financial statements. 

PROFESSIONAL AND OTHER SERVICES 

These services represent our next most signifi- 
cant area of expenditure, at close to 10% of total 
expenditures. Such costs were virtually the same as 
the previous year after several years of significant 
increases. These services include both contract 
professionals and contract CA firms. 

We continue to have to rely on contract profes- 
sionals to meet our legislated responsibilities, 
given more complex work and tighter deadlines for 
finalizing the financial-statement audits of Crown 
agencies and the province. We also believe that 
using more contract staff to fill temporary needs is 
a prudent approach to staffing, particularly during 
uncertain economic times, in that it provides more 
flexibility and less disruption if significant in-year 
cuts to our budget are requested. Also, even during 
the economic downturn it has remained difficult 
for us to reach our approved full complement given 
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our uncompetitive salary levels, particularly for 
professionals with several years of post-qualifying 
experience. Furthermore, after two years of budget 
freezes we can no longer afford to staff up to our 
approved complement of 117 staff (even if we could 
attract them at the salaries we are able to offer), 
and employing contract staff has proven a cost- 
effective way of helping to address this. 

We continue to incur higher contract costs for 
the CA firms we work with because of the higher 
salaries they pay their staff and the additional 
hours required to implement ongoing changes to 
accounting and assurance standards. However, 
these cost increases were partly offset this year by 
savings in provincial sales taxes with the introduc- 
tion of the refundable harmonized sales tax effect- 
ive July 1, 2010. 

RENT 

Our costs for accommodation were slightly lower 
than last year, owing primarily to a decline in 
building operating costs, particularly utilities. 
Accommodation costs declined as a percentage of 
total spending and are expected to decline slightly 
in the future as the result of the Office's successful 
negotiation of a rent reduction as part of the lease 
renewal terms commencing in fall 2011. 

TRAVEL AND COMMUNICATIONS 

Our travel and communications costs declined more 
than 6% from last year. This year the value-for- 
money audits we selected generally required less 
travel relative to the extensive broader-public-sector 
work completed last year, particularly in hospitals. 



This year only three of the audits required extensive 
travel: Forest Management Program, Supportive 
Services for People with Disabilities, and Student 
Success Initiatives. The introduction of the refund- 
able harmonized sales tax also contributed to the 
decline. In general, though, since the expansion of 
our value-for-money-audit mandate to the broader 
public sector, we have been incurring significantly 
more travel costs. 

OTHER 

Other costs include asset amortization, supplies 
and equipment maintenance, training, and statu- 
tory expenses. Such costs were virtually the same 
as last year, although the individual components 
fluctuated. Increases included $38,000 associ- 
ated with additional expert advisory services 
required to administer the Government Advertising 
Act, 2004, $16,000 relating to higher equipment 
amortization owing to prior investments in com- 
puter and leasehold improvements, and $12,000 
for the transfer payment to help cover additional 
training costs being shared by all Canadian legisla- 
tive audit offices on a collaborative basis. These 
increases were more than offset by reductions in 
expenditures on training ($24,000), owing mostly 
to the timing of course offerings relative to last 
year; statutory services ($22,000), owing to less 
special-audit activity; the Auditor General's salary 
($15,000), owing to salary restraint legislation; 
and supplies and equipment ($8,000). Again, the 
introduction of the refundable harmonized sales 
tax also contributed to the slight overall decline in 
other costs. 
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MANAGEMENT'S RESPONSIBILITY FOR FINANCIAL STATEMENTS 



The accompanying financial statements of the Office of the Auditor General for the year 
ended March 31, 201 1 are the responsibility of management of the Office. Management has 
prepared the financial statements to comply with the Auditor General Act and with Canadian 
generally accepted accounting principles. 

Management maintains a system of internal controls that provides reasonable assurance that 
transactions are appropriately authorized, assets are adequately safeguarded, appropriations 
are not exceeded, and the financial information contained in these financial statements is 
reliable and accurate. 

The financial statements have been audited by the firm of Adams & Miles LLP, Chartered 
Accountants. Their report to the Board of Internal Economy, stating the scope of their 
examination and opinion on the financial statements, appears on the following page. 




Jim McCarter, FC A Gary R. Peal 1, CA 

Auditor General Deputy Auditor General 



Box 105, 15th Floor 
20 Dundas Slreet West 
Toronto, Ontario 
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416-327-2381 
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20, rue Dundas ouest 
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Tel 416 502.2201 Te! 905 459.5605 
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INDEPENDENT AUDITORS' REPORT 

To the Board of Internal Economy of 
Legislative Assembly of Ontario 

We have audited the accompanying financial statements of the Office of the Auditor General of 
Ontario, which comprise the statement of financial position as at March 31 , 201 1 and the 
statements of operations and accumulated deficit and cash flows for the year then ended, and a 
summary of significant accounting policies and other explanatory information. 

Management's Responsibility for the Financial Statements 

Management is responsible for the preparation and fair presentation of these financial statements 
in accordance with Canadian generally accepted accounting principles, and for such internal 
control as management determines is necessary to enable the preparation of financial statements 
that are free from material misstatement, whether due to fraud or error. 

Auditors' Responsibility 

Our responsibility is to express an opinion on these financial statements based on our audit. We 
conducted our audit in accordance with Canadian generally accepted auditing standards. Those 
standards require that we comply with ethical requirements and plan and perform the audit to 
obtain reasonable assurance about whether the financial statements are free of material 
misstatement. 

An audit involves performing procedures to obtain audit evidence about the amounts and 
disclosures in the financial statements. The procedures selected depend on our judgment, 
including the assessment of the risks of material misstatement of the financial statements, whether 
due to fraud or error. In making those risk assessments, we consider internal control relevant to 
the preparation and fair presentation of the financial statements in order to design audit 
procedures that are appropriate in the circumstances, but not for the purpose of expressing an 
opinion on the effectiveness of the Office of the Auditor General of Ontario's internal control. An 
audit also includes evaluating the appropriateness of accounting policies used and the 
reasonableness of accounting estimates made by management, as well as evaluating the overall 
presentation of the financial statements. 

We believe that the audit evidence we have obtained is sufficient and appropriate to provide a 
basis for our audit opinion. 

Opinion 

In our opinion, the financial statements present fairly, in all material respects, the financial position 
of the Office of the Auditor General of Ontario as at March 31, 2011, and the results of its 
operations and its cash flows for the year then ended. 

Addms & MiCes LUP 



Chartered Accountants 
Licensed Public Accountants 

Toronto, Canada 
August 23, 2011 



1926 - 2011 

CELEBRATING 

85 YEARS 



www.adamsmiles.com 



An independent firm associated 
with AGN International Ltd. 
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Office of the Auditor General of Ontario 

Statement of Financial Position 
As at March 31, 2011 



Assets 

Current 
Cash 

Harmonized sales taxes recoverable 
Due from Consolidated Revenue Fund 



Capital assets (Note 4) 
Total assets 

Liabilities 

Accounts payable and accrued liabilities 

Accrued employee benefits obligation [Note 5(B)] 



Net accumulated deficit 

Investment in capital assets (Note 4) 

Accumulated deficit related to employee future benefits [Note 2(B)] 

Total liabilities and net accumulated deficit 
Commitments (Note 6) 

See accompanying notes to financial statements. 



2011 


2010 


$ 


$ 


500 170 


370 802 


128,927 




49,194 


754,098 


678,291 


1,124,900 


501,904 


540,543 


1,180,195 


1,665,443 


1,535,291 


1,920,900 


1,988,000 


1,922,000 


3,523,291 


3,842,900 


501,904 


540,543 


(2,845,000) 


(2,718,000) 


(2,343,096) 


(2,177,457) 


1,180,195 


1,665,443 



Approved by the Office of the Auditor General of Ontario: 





Jim McCarter, FCA 
Auditor General 



Gary Peall, CA 
Deputy Auditor General 
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Office of the Auditor General of Ontario 



Statement of Operations and Accumulated Deficit 
For the Year Ended March 31, 2011 





2011 


2011 


2010 




Budget 


Actual 


Actual 




(Unaudited) 








$ 


$ 


$ 


Expenses 








Salaries and wages 


9,755,400 


9,245,160 


8,870,759 


Employee benefits (Note 5) 


2,041,200 


1,987,895 


1,990,880 


Office rent 


1,062,400 


1,035,522 


1,068,789 


Professional and other services 


1,729,500 


1,490,944 


1,489,375 


Amortization of capital assets 




339,316 


323,386 


Travel and communication 


a 1 o onn 

41o,800 


o o i on 1 
33 t,6vl 


ocn no a 
359,934 


Training and development 


386,600 


130,700 


154,525 


Supplies and equipment 


377,500 


136,574 


143,734 


Transfer payment: CCAF-FCVI Inc. 


50 000 


61,775 


50 000 


Statutory expenses: Auditor General Act 


222,700 


229,147 


243,831 


Government Advertising Act 


30,000 


65,060 




Statutory services 


150,000 


108,434 


130,754 


Total expenses (Note 8) 


16,224,100 


15,167,828 


14,853,191 


Revenue 








Consolidated Revenue Fund - Voted appropriation [Note 2(B)] 


16,224,100 


16,224,100 


16,224,100 


Excess of appropriation over expenses 




1,056,272 


1,370,909 


Less: returned to the Province [Note 2(B)] 




1,221,911 


1,498,426 


Net operations deficiency 




(165,639) 


(127,517) 


Accumulated deficit, beginning of year 




(2,177,457) 


(2,049,940) 


Accumulated deficit, end of year 




(2,343,096) 


(2,177,457) 



See accompanying notes to financial statements. 
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Office of the Auditor General of Ontario 



Statement of Cash Flows 

For the Year Ended March 31, 2011 



NET INFLOW (OUTFLOW) OF CASH RELATED TO THE 
FOLLOWING ACTIVITIES 


2011 

$ 


2010 

$ 


Cash flows from operating activities 

iNei operations aenciency 
Amortization of capital assets 
Accrued employee benefits obligation 


339,316 
66,000 


[LZ f f Oll) 

323,386 
(75,000) 




239,677 


120,869 


Changes in non-cash working capital 

Decrease (increase) in due from Consolidated Revenue Fund 
Increase in Harmonized sales taxes recoverable 

Inrrpacp /HprrpaQp\ in arrmintc nauahlp anrl arrniprl liahilitipQ 
iiiuicdoc ^ucLicdoc^ hi duuuuiiLo |Jdyduic diiu duuiucu iiauiiiuco 


704,904 
(128,927) 
HSR RflQ'l 


(90,949) 




190,368 


239,496 


Investing activities 

Purchase of capital assets 


(300,677) 


(282,869) 


Net increase in cash position 


129,368 


77,496 


Cash position, beginning of year 


370,802 


293,306 


Cash position, end of year 


500,170 


370,802 



See accompanying notes to financial statements. 
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Notes to Financial Statements 
For the Year Ended March 31, 2011 



1. Nature of Operations 



In accordance with the provisions of the Auditor General Act and various other statutes and authorities, the 
Auditor General conducts independent audits of government programs, of institutions in the broader public 
sector that receive government grants, and of the fairness of the financial statements of the Province and 
numerous agencies of the Crown. In doing so, the Office of the Auditor General promotes accountability and 
value-for-money in government operations and in broader public sector organizations. 

Additionally, under the Government Advertising Act, 2004, the Auditor General is required to review specified 
types of advertising, printed matter or reviewable messages proposed by government offices to determine 
whether they meet the standards required by the Act. 

Under both Acts, the Auditor General reports directly to the Legislative Assembly. 

As required by the Fiscal Transparency and Accountability Act, 2004, the Auditor General has recently reviewed 
and reported on the 2011 Pre-Election Report prepared by the Ministry of Finance. 

2. Significant Accounting Policies 

The financial statements have been prepared in accordance with Canadian generally accepted accounting 
principles. The significant accounting policies are as follows: 

(A) ACCRUAL BASIS 

These financial statements are accounted for on an accrual basis whereby expenses are recognized in the fiscal 
year that the events giving rise to the expense occur and resources are consumed. 

(B) VOTED APPROPRIATIONS 

The Office is funded through annual voted appropriations from the Province of Ontario. Unspent appropriations 
are returned to the Province's Consolidated Revenue Fund each year. As the voted appropriation is prepared on a 
modified cash basis, an excess or deficiency of revenue over expenses arises from the application of accrual 
accounting, including the capitalization and amortization of capital assets and the recognition of employee 
benefit costs earned to date but that will be funded from future appropriations. 

(C) CAPITAL ASSETS 

Capital assets are recorded at historical cost less accumulated amortization. Amortization of capital assets is 
recorded on the straight-line method over the estimated useful lives of the assets as follows: 



Computer hardware 
Computer software 
Furniture and fixtures 



3 years 
3 years 
5 years 



Leasehold improvements 



The remaining term of the lease 
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Notes to Financial Statements 
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2. Significant Accounting Policies (Continued) 

(D) FINANCIAL INSTRUMENTS 

The Office's financial instruments consist of cash, due from Consolidated Revenue Fund, accounts payable and 
accrued liabilities, and accrued employee benefits obligation. Under Canadian generally accepted accounting 
principles, financial instruments are classified into one of five categories - available-for-sale, held-for-trading, 
held-to-maturity, loans and receivables, or other financial liabilities. The Office classifies its financial assets and 
liabilities as follows: 

• Cash is classified as held for trading and is recorded at fair value. 

• Due from Consolidated Revenue Fund is classified as loans and receivables and is valued at cost which 
approximates fair value given its short term nature. 

• Accounts payable and accrued liabilities are classified as other financial liabilities and are recorded at cost 
which approximate fair value given their short term maturities. 

• The accrued employee benefits obligation is classified as another financial liability and is recorded at cost 
based on the entitlements earned by employees up to March 31, 2011. A fair value estimate based on 
actuarial assumptions about when these benefits will actually be paid has not been made as it is not expected 
that there would be a significant difference from the recorded amount. 

It is management's opinion that the Office is not exposed to any interest rate, currency, liquidity or credit risk 
arising from its financial instruments due to their nature. 

(E) USE OF ESTIMATES 

The preparation of financial statements in accordance with Canadian generally accepted accounting principles 
requires management to make estimates and assumptions that affect the reported amounts of assets and 
liabilities at the date of the financial statements, and the reported amounts of revenue and expenses during the 
reporting period. Actual results could differ from management's best estimates as additional information 
becomes available in the future. 

3. Accounting Policy Transition 

In 2009, the Public Sector Accounting Board approved an amendment to "Introduction to Public Sector 
Accounting Standards". The amendment allowed organizations classified as an Other Government Organization 
to adopt either the Public Sector Accounting Handbook or International Financial Reporting Standards ("IFRS") 
for publicly accountable entities as the primary source of Canadian generally accepted accounting principles 
beginning on or after January 1, 2011. Taking into consideration the nature of the Office and its users, 
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3. Accounting Policy Transition (Continued) 

management has determined that Public Sector Accounting Standards are a more suitable accounting standard 
to follow. The Office is planning to adopt Public Sector Accounting Standards for fiscal year 2011/12. The 
impact on the financial statements is not expected to be significant. 



4. Capital Assets 







2011 




2010 






Accumulated 


Net Book 


Net Book 




Cost 


Amortization 


Value 


Value 




$ 


$ 


$ 


$ 


Computer hardware 


597,133 


397,966 


199,167 


248,605 


Computer software 


340,833 


147,169 


193,664 


90,469 


Furniture and fixtures 


378,491 


294,185 


84,306 


127,167 


Leasehold improvements 


235,868 


211,101 


24,767 


74,302 




1,552,325 


1,050,421 


501,904 


540,543 



Investment in capital assets represents the accumulated cost of capital assets less accumulated amortization and 
disposals. 

5. Obligation for Future Employee Benefits 

Although the Office's employees are not members of the Ontario Public Service, under provisions in the Auditor 
General Act, the Office's employees are entitled to the same benefits as Ontario Public Service employees. The 
future liability for benefits earned by the Office's employees is included in the estimated liability for all provincial 
employees that have earned these benefits and is recognized in the Province's consolidated financial statements. 
These benefits are accounted for as follows: 

(A) PENSION BENEFITS 

The Office's employees participate in the Public Service Pension Fund (PSPF) which is a defined benefit pension 
plan for employees of the Province and many provincial agencies. The Province of Ontario, which is the sole 
sponsor of the PSPF, determines the Office's annual payments to the fund. As the sponsor is responsible for 
ensuring that the pension funds are financially viable, any surpluses or unfunded liabilities arising from statutory 
actuarial funding valuations are not assets or obligations of the Office. The Office's required annual payments of 
$732,873 (2010 - $711,251), are included in employee benefits expense in the Statement of Operations and 
Accumulated Deficit. 
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5. Obligation for Future Employee Benefits (Continued) 

(B) ACCRUED EMPLOYEE BENEFITS OBLIGATION 

Although the costs of any legislated severance and unused vacation entitlements earned by employees are 
recognized by the Province when earned by eligible employees, these costs are also recognized in these financial 
statements. These costs for the year amounted to $231,000 (2010 - $229,000) and are included in employee 
benefits in the Statement of Operations and Accumulated Deficit. The total liability for these costs is reflected in 
the accrued employee benefits obligation, less any amounts payable within one year, which are included in 
accounts payable and accrued liabilities, as follows: 





2011 


2010 




$ 


$ 


Total liability for severance and vacation 


2,845,000 


2,718,000 


Less: Due within one year and included in 






accounts payable and accrued liabilities 


857,000 


796,000 


Accrued employee benefits obligation 


1,988,000 


1,922,000 



(C) OTHER NON-PENSION POST-EMPLOYMENT BENEFITS 

The cost of other non-pension post-retirement benefits is determined and funded on an ongoing basis by the 
Ontario Ministry of Government Services and accordingly is not included in these financial statements. 

6. Commitments 

The Office has an operating lease to rent premises which expires on October 31, 2021. The minimum rental 
commitment for the remaining term of the lease is as follows: 



$ 

2011- 12 507,800 

2012- 13 483,000 

2013- 14 488,400 

2014- 15 495,900 

2015- 16 501,300 

2016- 17 and beyond 2,920,800 
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7. Public Sector Salary Disclosure Act, 1996 

Section 3(5) of this Act requires disclosure of the salary and benefits paid to all Ontario public-sector employees 
earning an annual salary in excess of $100,000. This disclosure for the 2010 calendar year is as follows: 







Salary and 


Taxable 






Benefits Paid 


Benefits 


Name 


Position 


$ 


$ 


McCarter, Jim 


Auditor General 


254,174 


4,306 


Peall, Gary 


Deputy Auditor General 


172,985 


282 


Amodeo, Paul 


Director 


140,858 


222 


Bell, Laura 


Director 


122,582 


194 


Bordne, Walter 


Director 


131,000 


222 


Chagani, Gus 


Director 


122,582 


194 


Cheung, Andrew 


Director 


140,858 


222 


Chiu, Rudolph 


Director 


140,858 


222 


Fitzmaurice, Gerard 


Director 


144,145 


222 


Klein, Susan 


Director 


140,858 


222 


Mazzone, Vince 


Director 


140,858 


222 


McDowell, John 


Director 


144,145 


222 


Allan, Walter 


Audit Manager 


113,962 


179 


Brennan, Michael 


Audit Manager 


101,733 


149 


Carello, Teresa 


Audit Manager 


107,673 


173 


Chan, Sandy 


Audit Manager 


113,962 


179 


Cumbo, Wendy 


Audit Manager 


116,621 


179 


Gotsis, Vanna 


Audit Manager 


113,962 


179 


Herberg, Naomi 


Audit Manager 


113,962 


179 


MacNeil, Richard 


Audit Manager 


113,962 


179 


Pelow, William 


Audit Manager 


116,621 


179 


Rogers, Fraser 


Audit Manager 


105,986 


179 


Sin, Vivian 


Audit Manager 


108,473 


174 


Stavropoulos, Nick 


Audit Manager 


110,218 


203 


Tersigni, Anthony 


Audit Manager 


113,962 


179 


Tsikritsis, Emanuel 


Audit Manager 


104,618 


173 


Young, Denise 


Audit Manager 


116,621 


179 


Boer, Johannes 


Audit Supervisor 


104,157 


169 


Bove, Tino 


Audit Supervisor 


100,187 


166 


Davy, Howard 


Audit Supervisor 


104,157 


169 


Hancock, Mark 


Audit Supervisor 


104,157 


169 


Tepelenas, Ellen 


Audit Supervisor 


104,017 


167 


Wanchuk, Brian 


Audit Supervisor 


100,651 


169 


Yeung, Celia 


Audit Supervisor 


106,778 


169 


Wiebe, Annemarie 


Manager, Human Resources 


113,962 


179 
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8. Reconciliation to Public Accounts Volume 1 Basis of Presentation 

The Office's Statement of Expenses presented in Volume 1 of the Public Accounts of Ontario was prepared on a 
basis consistent with the accounting policies followed for the Province's financial statements, under which 
purchases of computers and software are expensed in the year of acquisition rather than being capitalized and 
amortized over their useful lives. Volume 1 also excludes the accrued employee future benefit costs recognized in 
these financial statements as well as in the Province's summary financial statements. A reconciliation of total 
expenses reported in Volume 1 to the total expenses reported in these financial statements is as follows: 





2011 

$ 


2010 

$ 


Total expenses per Public Accounts Volume 1 


15,002,189 


14,725,674 


purchase of capital assets 

amortization of capital assets 

change in accrued future employee benefit costs 


(300,677) 
339,316 
127,000 


(282,869) 
323,386 
87,000 




165,639 


127,517 


Total expenses per audited financial statements 


15,167,828 


14,853,191 



9. Management of Capital 

The Office's capital consists of cash on hand. In managing cash on hand the Office maintains sufficient funds to 
meet estimated cash requirements each month and requisitions the necessary amount from the Ministry of 
Finance on a monthly basis. The Office's bank account is pooled with other government accounts for cash 
management purposes in order to reduce the province's borrowing requirements and to earn interest at rates 
negotiated by the Ministry of Finance. Accordingly, the Office's capital is not at risk. 




Agencies of the Crown 



1. Agencies whose accounts are audited 
by the Auditor General 

Agricorp 

Algonquin Forestry Authority 
Cancer Care Ontario 

Centennial Centre of Science and Technology 
Chief Electoral Officer, Election Finances Act 
Election Fees and Expenses, Election Act 
Financial Services Commission of Ontario 
Grain Financial Protection Board, Funds for 

Producers of Grain Corn, Soybeans, Wheat, and 

Canola 

Investor Education Fund, Ontario Securities 

Commission 
Legal Aid Ontario 
Liquor Control Board of Ontario 
Livestock Financial Protection Board, Fund for 

Livestock Producers 
Northern Ontario Heritage Fund Corporation 
Office of the Assembly 
Office of the Children's Lawyer 
Office of the Environmental Commissioner 
Office of the Information and Privacy 

Commissioner 
Office of the Ombudsman 



Ontario Clean Water Agency (December 31) 1 

Ontario Development Corporation 

Ontario Educational Communications Authority 

Ontario Electricity Financial Corporation 

Ontario Energy Board 

Ontario Financing Authority 

Ontario Food Terminal Board 

Ontario Heritage Trust 

Ontario Immigrant Investor Corporation 

Ontario Media Development Corporation 

Ontario Mortgage and Housing Corporation 

Ontario Northland Transportation Commission 

Ontario Place Corporation (December 31) 1 

Ontario Racing Commission 

Ontario Realty Corporation 2 

Ontario Securities Commission 

Pension Benefits Guarantee Fund, Financial 

Services Commission of Ontario 
Province of Ontario Council for the Arts 
Provincial Advocate for Children and Youth 
Provincial Judges Pension Fund, Provincial Judges 

Pension Board 
Public Guardian and Trustee for the Province of 

Ontario 



1. Dates in parentheses indicate fiscal periods ending on a 
date other than March 31. 

2. Effective June 6, 2011, Ontario Realty Corporation, 
Stadium Corporation of Ontario Limited, and Ontario 
Infrastructure Projects Corporation merged and are now 
known as Ontario Infrastructure and Land Corporation. 



453 
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2. Agencies whose accounts are audited 
by another auditor under the direction of 
the Auditor General 

Motor Vehicle Accident Claims Fund 
Niagara Parks Commission (October 31) 1 
Ontario Mental Health Foundation 
St. Lawrence Parks Commission 
Workplace Safety and Insurance Board 
(December 31) 1 



1. Dates in parentheses indicate fiscal periods ending on a 
date other than March 31. 



Note: 

The following changes were made during the 2010/11 
fiscal year: 
Deletion: 

North Pickering Development Corporation 
Toronto Area Transit Operating Authority 




Crown-controlled 
Corporations 



Corporations whose accounts are 
audited by an auditor other than the 
Auditor General, with full access by the 
Auditor General to audit reports, working 
papers, and other related documents 

Agricultural Research Institute of Ontario 
Board of Funeral Services 
Brock University Foundation 
Central Community Care Access Centre 
Central East Community Care Access Centre 
Central East Local Health Integration Network 
Central Local Health Integration Network 
Central West Community Care Access Centre 
Central West Local Health Integration Network 
Champlain Community Care Access Centre 
Champlain Local Health Integration Network 
Deposit Insurance Corporation of Ontario 

(December 31) 1 
Echo: Improving Women's Health in Ontario 
Education Quality and Accountability Office 
eHealth Ontario 

Erie St. Clair Community Care Access Centre 
Erie St. Clair Local Health Integration Network 
Foundation at Queen's University at Kingston 
Hamilton Niagara Haldimand Brant Community 

Care Access Centre 
Hamilton Niagara Haldimand Brant Local Health 

Integration Network 



HealthForceOntario Marketing and Recruitment 
Agency 

Higher Education Quality Council of Ontario 
Human Rights Legal Support Centre 
Hydro One Inc. (December 31) 1 
Independent Electricity System Operator 

(December 31) 1 
McMaster University Foundation 
McMichael Canadian Art Collection 
Metrolinx 

Metropolitan Toronto Convention Centre 
Corporation 

Mississauga Halton Community Care Access Centre 
Mississauga Halton Local Health Integration 
Network 

Municipal Property Assessment Corporation 
North East Community Care Access Centre 
North East Local Health Integration Network 
North Simcoe Muskoka Community Care Access 
Centre 

North Simcoe Muskoka Local Health Integration 
Network 

North West Community Care Access Centre 
North West Local Health Integration Network 
Ontario Agency for Health Protection and 

Promotion 
Ontario Capital Growth Corporation 
Ontario French-language Educational 

Communications Authority 



1. Dates in parentheses indicate fiscal periods ending on a 
date other than March 31. 
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Ontario Health Quality Council 
Ontario Infrastructure Projects Corporation 2 
Ontario Lottery and Gaming Corporation 
Ontario Mortgage Corporation 
Ontario Pension Board (December 31) 1 
Ontario Power Authority (December 31) 1 
Ontario Power Generation Inc. (December 31) 1 
Ontario Tourism Marketing Partnership 

Corporation 
Ontario Trillium Foundation 
Ottawa Convention Centre Corporation 
Owen Sound Transportation Company Limited 
Royal Ontario Museum 
Science North 

South East Community Care Access Centre 
South East Local Health Integration Network 



South West Community Care Access Centre 
South West Local Health Integration Network 
Stadium Corporation of Ontario Limited 2 
Toronto Central Community Care Access Centre 
Toronto Central Local Health Integration Network 
Toronto Islands Residential Community Trust 

Corporation 
Toronto Waterfront Revitalization Corporation 
Trent University Foundation 
Trillium Gift of Life Network 
University of Ottawa Foundation 
Walkerton Clean Water Centre 
Waterfront Regeneration Trust Agency 
Waterloo Wellington Community Care Access Centre 
Waterloo Wellington Local Health Integration 

Network 



1. Dates in parentheses indicate fiscal periods ending on a 
date other than March 31. 

2. Effective June 6, 2011, Ontario Realty Corporation, 
Stadium Corporation of Ontario Limited, and Ontario 
Infrastructure Projects Corporation merged and are now 
known as Ontario Infrastructure and Land Corporation. 

Note: 

The following changes were made during the 2010/11 
fiscal year: 
Deletions: 

Art Gallery of Ontario Crown Foundation 

Canadian Opera Company Crown Foundation 

Canadian Stage Company Crown Foundation 

National Ballet of Canada Crown Foundation 

Northern Ontario Grow Bonds Corporation 

Ontario Foundation for the Arts 

Royal Botanical Gardens Crown Foundation 

Royal Ontario Museum Crown Foundation 

Shaw Festival Crown Foundation 

Stratford Festival Crown Foundation 

Toronto Symphony Orchestra Crown Foundation 




Treasury Board Orders 



Under subsection 12(2) (e) of the Auditor General 
Act, the Auditor General is required to annually 
report all orders of the Treasury Board made to 
authorize payments in excess of appropriations, 
stating the date of each order, the amount author- 
ized, and the amount expended. These are outlined 



in the following table. Although ministries may 
track expenditures related to these orders in more 
detail by creating accounts at the sub-vote and item 
level, this schedule summarizes such expenditures 
at the vote and item level. 



Ministry 



Date of Orde 



uthorized ($) Expended ($) 



Aboriginal Affairs 



Agriculture, Food and Rural Affairs 



Attorney General 



Cabinet Office 



Children and Youth Services 



Jun. 17, 2010 


2,800,000 


2,800,000 


Jun. 17, 2010 


500,000 


— 


Aug. 23, 2010 


3,640,000 


1,467,200 


Oct. 21, 2010 


5,195,100 


5,089,000 


Oct. 21, 2010 


523,000 


— 


Feb. 17, 2011 


1,060,000 




Feb. 17, 2011 


5,090,000 




Apr. 14, 2011 


290,000 






19,098,100 


9,356,200 


Jul. 28, 2010 


9,000,000 




Mar. 22, 2011 


28,000,000 


13,073,844 




37,000,000 


13,073,844 


Dec. 2, 2010 


450,000 




Feb. 17, 2011 


2,528,700 


2,528,700 


Mar. 29, 2011 


4,832,300 


1,164,099 


Mar. 29, 2011 


200,500 


409 




8,011,500 


3,693,208 


Jun. 17, 2010 


770,000 




Aug. 5, 2010 


200,000 




Aug. 12, 2010 


800,000 






1,770,000 




Nov. 15, 2010 


397,000 




Feb. 17, 2011 


30,300,000 


30,300,000 


Mar. 3, 2011 


4,500,000 


3,577,800 


Mar. 3, 2011 


10,600,000 


10,600,000 


Mar. 18, 2011 


31,304,000 


16,927,921 


Apr. 14, 2011 


2,269,800 






79,370,800 


61,405,721 
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Authorized ($) 


Expended ($) 










Citizenship and Immigration 


Mar. 31, 2011 


820,300 


724,020 










Community and Social Services 


Mar. 24, 2011 


800,000 


- 


Community Safety and Correctional Services 


Jul. 7, 2010 


2,600,000 


_ 




Mar. 29, 2011 


20,258,600 


17,810,856 




Apr. 14, 2011 


65,443,600 


63,503,597 






88,302,200 


81,314,453 


Economic Development and Trade 


Sep. 16, 2010 


31,800,000 


_ 




Nov. 29, 2010 


1,975,000 


_ 




Dec. 2, 2010 


700,000 


_ 






34,475,000 





Education 


Apr. 15, 2010 


850,000 


850,000 




Sep. 16, 2010 


20,861,200 


13,983,065 




Nov. 18, 2010 


835,812,700 


834,772,700 




Mar. 29, 2011 


4,249,400 






Mar. 30, 2011 


1,193,600 


_ 




Apr. 11, 2011 


5,279,000 


2,073,673 




Apr. 14, 2011 


37,409,600 


_ 






905,655,500 


851,679,438 


Energy and Infrastructure* 


Sep. 16, 2010 


39,245,000 







Oct. 21, 2010 


25,300,000 


25,300,000 




Jan. 24, 2011 


300,000,000 


299,826,592 




Jan. 27, 2011 


85,100,000 


85,100,000 




Feb. 17, 2011 


5,000,000 


_ 




Mar. 29, 2011 


13,983,000 


_ 




Apr. 11, 2011 


2,840,900 


2,840,900 




Apr. 14, 2011 


2,000,200 






Apr. 14, 2011 


26,820,000 


24,420,000 




May 19, 2011 


108,270,000 


106,828,837 






608,559,100 


544,316,329 










Environment 


Feb. 17, 2011 


21,915,900 


21,876,941 


Finance 


Aug. 23, 2010 


470,200 







Aug. 23, 2010 


12,000,000 







Sep. 13, 2010 


234,500 


_ 




Sep. 16, 2010 


15,000,000 


_ 




Feb. 17, 2011 


18,627,200 


12,039,672 




Mar. 29, 2011 


30,031,700 






Mar. 29, 2011 


62,431,600 






Mar. 29, 2011 


450,000,000 


447,000,000 




Apr. 11, 2011 


1,452,100 








590,247,300 


459,039,672 



* Now two separate ministries-Ministry of Energy and Ministry of Infrastructure 



Exhibit 3 



Ministry 



Government Services 



Health and Long-Term Care 



Health Promotion and Sport 



Labour 



Municipal Affairs and Housing 



Natural Resources 



Northern Development, Mines and Forestry 



Date of Order 


Authorized ($) 


Expended ($) 


Jun. 17, 2010 


2,405,000 




Jun. 17, 2010 


2,383,000 




Nov. 18, 2010 


5,000,000 




Dec. 2, 2010 


4,506,600 


4,506,600 


Dec. 2, 2010 


2,221,200 


2,221,200 


Dec. 16, 2010 


152,223,000 


130,631,561 


Jan. 27, 2011 


495,000 


493,552 


Feb. 17, 2011 


3,773,400 


3,709,675 


Feb. 28, 2011 


10,822,300 


7,658,554 


Apr. 8, 2011 


900,000 




Apr. 11, 2011 


208,700 




Jun. 16, 2011 


6,518,800 


6,387,886 




191,457,000 


155,609,028 


Aug. 23, 2010 


4,964,800 




Oct. 15, 2010 


3,442,300 


3,373,858 


Dec. 2, 2010 


1,602,200 


1,602,200 


Dec. 2, 2010 


5,860,000 


4,308,317 


Jan. 27, 2011 


8,925,900 


6,666,402 


Feb. 17, 2011 


1,324,348,000 


1,301,047,904 


Apr. 14, 2011 


37,567,900 


30,016,652 




1,386,711,100 


1,347,015,333 


Sep. 16, 2010 


6,836,800 




Mar. 23, 2011 


1,150,000 


378,221 


Apr. 14, 2011 


175,000 






8,161,800 


378,221 








Feb. 3, 2011 


1,155,700 


625,285 


Jun. 17, 2010 


1,000,000 


1,000,000 


Sep. 16, 2010 


199,300 


199,300 


Jan. 27, 2011 


25,000,000 


21,120,000 


Feb. 17, 2011 


6,683,900 


5,272,185 


Mar. 3, 2011 


20,000,000 


19,466,763 


May 19, 2011 


1,122,000 






54,005,200 


47,058,248 


Jul. 7, 2010 


57,100,000 


34,999,328 


Sep. 16, 2010 


22,942,000 


11,958,642 


Dec. 2, 2010 


3,500,000 


3,500,000 


Dec. 2, 2010 


1,000,000 


1,000,000 


Apr. 14, 2011 


9,117,600 


7,897,497 




93,659,600 


59,355,467 


Sep. 16, 2010 


18,586,000 




Oct. 21, 2010 


8,800,000 


8,628,783 


Dec. 2, 2010 


1,500,000 


1,500,000 


Jan. 27, 2011 


1,991,700 


1,991,700 
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Ministry 


Date of Order 


Authorized ($) 


Expended ($) 1 


Northern Development, Mines and Forestry (continued) 


Mar. 16, 2011 


5,735,000 


1,338,430 




Apr. 11, 2011 


1,070,700 






Apr. 14, 2011 


11,606,000 


10,813,629 






49,289,400 


24,272,542 










Office of Francophone Affairs 


Nov. 18, 2010 


465,000 




Revenue 


Jun. 17, 2010 


20,000,000 


6,366,477 




Jun. 17, 2010 


2,645,900 


2,480,000 




Aug. 23, 2010 


40,869,800 


2,930,115 




Sep. 16, 2010 


1,200,000 






Jan. 24, 2011 


3,315,600 






Feb. 17, 2011 


5,100,000 






Apr. 11, 2011 


1,490,900 








74,622,200 


11,776,592 



Tourism and Culture 



Jun. 17, 2010 
Aug. 23, 2010 
Sep. 16, 2010 
Sep. 16, 2010 
Nov. 18, 2010 
Apr. 4, 2011 
Apr. 11, 2011 
Apr. 15, 2011 



Mar. 30, 2011 
Apr. 11, 2011 



3,950,000 
1,000,000 
29,017,400 
10,000,000 
2,000,000 
4,926,200 
65,700 
10,400,000 



1,600,000 
758,000 



3,950,000 
1,000,000 
25,185,953 
2,226,629 
2,000,000 
4,664,642 







61,359,300 


39,027,224 


Training, Colleges and Universities 


Sep. 16, 2010 


7,760,000 






Feb. 10, 2011 


3,497,000 


404,247 






11,257,000 


404,247 


Transportation 


Sep. 16, 2010 


124,450,000 


9,547,517 




Oct. 26, 2010 


5,000,000 


4,994,540 



169,930 



131,808,000 



14,711,987 



Total Treasury Board Orders 



4,459,977,000 



3,746,714,000 



